LLMs very much operate on the level of text. That text can represent ideas, but it can also directly be a copyrighted work.

I am however very interested in the novel interpretation of copyright that says that you can do whatever as long as your compression is lossy.