The point is that the statement "LLMs should just cite their sources, what's the problem" is nonsensical, and the reason it's nonsense has to do with how LLMs actually work.
LLMs can cite sources as well as any human, that is with a non-trivial error rate.
LLMs are shit for a lot of things but the problems are with the quality of the output whether they work by magic, soul-bending, matrix multiplication, or whatever is irrelevant.