Such a massive theft of labor will have negative consequences that we all have to live with.
I've noticed that in "programming" forums, questions have devolved.
In the "before times", you'd get a mix of simple-medium-hard-extreme questions.
Google and StackOverflow made most of the medium and hard questions disappear.
Now with LLMs, all the higher difficulty levels except "beyond extreme" have vanished, leaving only the stupidly difficult questions that nobody can answer -- and hence doesn't contribute to collective knowledge, including the AI's knowledge.
Conversely, there is a new level of "below simple" questions by people that I dare say are too stupid to even attempt to become software developers. The type of questions that can could always have been answered with just Google, and are "beneath" both AIs and humans. These are worthless questions along the lines of "my computer not work, plz halp!?" followed by "I can't reboot it!" as if that makes any sense.
Again, that contributes nothing of value to the zeitgeist.
Worse, because the effort required to spin up a blog is trivial now thanks to AI and people are getting desperate for work, I'm seeing a mountain of "slop coding blogs" that are either 100% low-cost-AI output full of errors, or organic human-origin slop... full of errors. No longer are blogs written by the "best of the best" to disseminate their hard earned knowledge, they're now an alternative to a resume and produced in bulk by the most junior developers hoping to land a job through "exposure".
This also massively dilutes the last remaining source of good public information to draw. Google searches are useless now because finds nothing but endless SEO garbage.
What's going to happen in 5 or 10 years from now, when all of the pre-AI knowledge[1] is stale and out of date, making it irrelevant, but there's no new knowledge being publicly produced to train on?
Will programming languages, tools, and frameworks ossify because new languages are "post-AI" and hence untrainable due to a lack of StackOverflow questions and answers?
Will AI labs build their own private "AI members only" version of StackOverflow, building up their internal knowledgebases for training, sharing this with nobody else?
Will humans be paid to solve problems for the sole purpose of generating AI training materials? Is this the future of knowledge work!?
You can't steal something that's infinitely duplicatable. Also, AI training is fair use, not that I believe it should have to be.
There are many ways to steal labor, violating copyright is one such way, but not all copyright violations is labor theft, nor does labor theft require copyright to be violated.