It's a bit like saying we can't stop music piracy, now that Napster exists.
Remember the naive rallying cry among those who thought everyone should have the right to all music, without any compensation for the artist?
"Information wants to be free!" (https://www.theguardian.com/music/2013/feb/24/napster-music-...)
Napster was a peer-to-peer file sharing application. It originally launched on June 1, 1999, with an emphasis on digital audio file distribution. ... It ceased operations in 2001 after losing a wave of lawsuits and filed for bankruptcy in June 2002.
Use of the output of systems like Copilot or Stable Diffusion becomes a violation of copyright.
The weight tensors are illegal to possess, just like it's illegal to possess leaked Intel source code. The weights are like distilled intellectual property. You're distributing an enormous body of other people's work, to enable derivative work without attribution? Huge harm to society, make it illegal.
If you use the art in your product, on your website, etc., you risk legal action. Just like if I publish your album on my website. Illegal.
The companies that train these systems can't distribute them without risking legal action. So they won't do it. It's expensive to train these models. When it's illegal, the criminals will have to pay for the GPU time.
It will always exist in the black-market underground, but the civilized world makes it illegal.
That's where this is going, I hope. Best case scenario.