But the output of the LLM is trained on a synthesis of multiple inputs. Even if every input has information loss, new and novel combinations can be created. I imagine it’s the same for humans since we have lossy memory.
Humans have lossy memory but we constantly engage with the real world so it doesn't matter. The key here is we engage with the real world on a regular basis.
The key idea is that you can improve a low fidelity representation by training on a higher fidelity representation. But you can not get a higher fidelity representation from a low fidelity representation. Any inference is just making things up and likely to be wrong, so it is more like to just have less details.