>CD-ROM burners
They can be used in an illegal way if used to copy copyrighted material, yes.
They can be used in an illegal way if used to copy copyrighted material, yes.
You do not have to produce an exact copy of something to violate copyright, and I think anything the LLM outputs is violating copyright for everything it has ever trained on, unless the operator (the person operating the LLM and/or the person prompting it) has rights to that content.
I personally am not against LLMs training on things the operator has rights to, and even training on copyrighted things, but I am against it laundering those things back out and claiming it's legal.