To explain this further: OpenAI et al. (as commercial products) are being trained on content that is published under licenses that allow non-commercial use only. Do those systems respect these licenses? It doesn't look like that. "AI companies" need to stick to laws but as nobody is able to look inside their blackboxes, we can't make sure they follow the law. That's where legislation like this comes from.
and that's bad because?
I would see the point if they were training on my private data I entrusted to somebody and they illegally obtained it without my permission. Are they doing that?
Also neither company seem to have much regard for user privacy.
As long as copyright is here; it is expected big players are to be bound by it to the same degree they push legal systems to bind the little guy.
What you get instead, is the big guy pilfering the little guys under the justification that "it's different when we do it, and if you challenge us, I'll put my subsidized legal department to work burying you."
Copyright needing significant overhaul or abolition doesn't detract from that state of affairs, I hope we can agree?