Had they never existed, it had likely not made a dent to the AI development - completely like believing that had they been twice as productive, it had likely neither made a dent to the quality of LLMs.
LLMs are (apparently) massively used to get information about topics in the real world. Novels aren't going to be much help there. Journalism, particularly in written form, provides a fount of facts presented from different angles, as well as opinions, and it was all there free for the taking…
Wikipedia provides the scantest summary of that, fora and social media give you banter, fake news, summaries of news, and a whole lot of shaky opinions, at best. Novels give you the foundations of language, but in terms of knowledge nothing much beyond what the novel is about.
Imagine if all info about Facebook came from Facebook...
But even taking it literally, isn't that one of the things LLMs could actually do? You're essentially asking how a text generator could generate text. The real question is whether the questions would be any good, but the answer isn't necessarily no.
You used to need them, because journalists had the distribution and the sources didn't. In a word of printed newspapers, you couldn't get your story distributed nationally (much less worldwide) without the help of a journalist, doubly so if you wanted to stay anonymous.
Nowadays, you just make a Substack and there's that.
See that recent expose on the Delve fraud as just one example. No journalists were harmed in the making of that article.
Start thinking!
Journalism is by definition a secondary source. (Notwithstanding edge cases like articles reporting directly on the news industry itself.)
If a journalist is on location covering a flood, for example, they are the primary source.
A journalist conducting an interview would also be a primary source.
NY Times is 0.06% of common crawl.
These news media outlets provide a drop in the ocean worth of information. Both qualitatively and quantitatively.
The news / media industry is really just trying to hold on to their lifeboat before inevitably becoming entirely irrelevant.
(I do find this sad, but it is like the reality - I can already now get considerably better journalism using LLMs than actual journalists - both click bait stuff and high quality stuff)
Instead, training attempts to sample more heavily from higher quality sources, with, I'm sure, a mix of manual and heuristic labeling.
their idioms would leak occasionally otherwise
And don't basically all those news sites allow google on purpose?
At least NYT is probably on the correct side of Sturgeon’s Law: https://en.wikipedia.org/wiki/Sturgeon%27s_law
You may get an inconvenient answer when you ask the question the other way around.
I'd imagine they already have a database of users such that if multiple people talk about a possibly true subject they can ask subject experts; users related to the subject for clarification and further information.
It is not like they paid anybody else for their contribution either.
It is just not more worth than anything else in the data sets.
Many publications put information on the internet for the first time, or curate it for the first time, or research a topic deeper than ever before. Someone - a thinking, feeling human - had to get out there and try restaurants, talk to people, pore through archives, read books, use products. Each of them contribute a little to what we know about the world.
I do this for a living. AI might soon put me out of work. It already more than halved my audience, using my own work. It's sickening to see people cheer for it because they have a bone to pick with certain websites. Eventually those websites will be gone, but so will the good ones that produced critical information.
Preventing new human generated text from being used by AI firms (without consent) seems like a valid strategy.
Modern LLMs are trained on a large percentage of synthetic data.
This sentiment is largely legacy (even though just a couple of years old).