The line we draw is "did a human creatively and intentionally produce the output" or "did a computer".
It doesn't matter that we've built a compression algorithm no one actually understands (which is what LLMs effectively are, a lossy compression algorithm, compressing their input into a model), the bar for copyrightable creativity was, and still is, human creativity. Which, by definition, a computer does not have, unless a human infuses it into the computer specifically (by for example using the computer to produce a specific copyrighted work).