I do think it’ll be interesting to see how our militaries handle the issues. Considering that is probably the one area our governments don’t want to be “behind”.
To me, this leads to another question: is it possible to get a GPT-4 quality model using only training data from major content providers?
In other words, what if OpenAI didn’t crawl the entire internet and only used data from Facebook, Twitter, Wikipedia, GitHub, NYT, Stackoverflow, and [list 100 more well known high quality data sources here]
If it’s possible to get to GPT-4 model quality with training data from a couple hundred centralized sources, then maybe it won’t have any impact at all (other than OpenAI needing to pay other companies to use their data for training data)
Or, if the long tail “every website on the internet” really is the secret to training LLM models, then that would be much more problematic.
It's not even down to the NYT and such: Think facebook, or reddit, suddenly being able to charge others for their massive amount of data, while they force anyone posting there to make their content usable for their own training to post there at all. Imagine a world where, say, Microsoft buys the times, just because it's better for them to not just have the training data, but to tell every competitor to get bent if they want a license. 'Google buys Disney for the training data'. If AI is that useful, and we make copyright that strong, what we might see is ailing media companies being bought out by tech.
Ultimately IP law is about the tradeoff of more original content and more usefulness for the world, and AI massively changes where the center of balance is. So I don't expect a ruling for the NYT to remain the law of the land for very long, just due to economic pressures.
Ultimately it doesn't matter which side people like aesthetically: If an AI trained with no regard for copyright is that much more useful than one that quotes every source and pays it some money, then piracy will ensure. If it doesn't really help, then copyright will keep being very strong.
It's not even a matter of countries, but hackers. Would Aaron Swartz, working on his own LLM, pay for copyright dues or not? Because there's enough people in tech with that mindset that we'll end up with LLMs trained by a group that thinks like he did.
We will have laws, people in places that aren't subjected to our laws, and pirates. It's now it always works.
If it is found to be breach of copyright then the berne convention comes into play which makes it hard(er) for international offerings to exist.
AI is only useful in so far as it's grounded in reality. If you ask it how tall the empire state building is, the answer has to be in some way derived from how tall it really is. Even creative writing is only useful in so far as how it's grounded in real life human experience. This is all to say that a bunch of people need to keep writing stuff about the real world for models to keep up with the real world.
I'd say the ideal scenario for US AI development is:
1) People keep making high quality data grounded in real world USA
2) That data is easily accessible for research purposes
3) That data is usable for commercial purposes (but might require more work than for research purposes)
A ruling for NYT here might keep good data more widely produced and available. Research legality is almost surely not going to be affected, so keeping data produced and available is key to keeping domestic research going strong and keeping talent here.
A ruling against NYT might mean all the new grounded-in-reality data goes behind closed doors. And even though the legality of research isn't in question, that would make the research harder. It would be easier to commercially use what you can find, but it could be garbage-in-garbage-out in the sense of quality data becoming hidden.
Ideally a marketplace would develop that can price each content fairly.
Probably none. They can just open a subsidiary there and siphon the same data to train their models.
Licensing the most prominent IP holders will get the vast majority of data secured.
These billion-dollar-funded efforts can comfortably chase down, say, hundreds of deals.
GPT-4 finished training g 18mo ago…plenty of time to clean up this mess, if there had been a will
2. Open door for everyone else whose content has been copied to demand payment.
3. Japan aside*, I'm not sure how a decisive victory for NYT fails to affect companies in other countries. Other countries will certainly take a US opinion into account. Furthermore, nothing whatsoever would prevent NYT from suing for infringement in France, Germany, etc.
A number of European countries are _much_ more comfortable with compulsory licensing than the US is.** I will not be surprised at all if Germany, Denmark, Sweden etc impose an AI-training license that companies have to pay to train or even use AI models. Could be based on employees, users, revenue, etc. They've been doing this for decades for other forms of mechanical reproduction, so it seems a reasonable path forward.
* I've seen reports that Japan has declared that it will not enforce copyright against organizations for training models on copyright-protected works. I don't know whether this means that _everything_ is fair game in Japan-- it could mean that training won't be enforced, but commercial use of models trained on protected works could be. I think it's somewhat harder to argue against training a model than it is against leveraging a trained model for commercial purposes.
** AFAIK, the only use for compulsory licensing in the US is musical performance.
4. "Surely one can't get licenses"... collective licensing would be a path forward for which there's a little more history in the US than there is for compulsory licensing. In a collective licensing model, OpenAI/Microsoft/Company X would be able to purchase a license priced by revenue/employees/users etc, and funds would be distributed to participating copyright holders.
For that to work, you need some kind of agency that has a ton of significant copyright holders. If some agency were able to get to a critical mass , that could happen without Congressional action.
Congress could also step into the problem in a couple of ways. They could wave a wand and make the problem go away for OpenAI/Microsoft/everyone else, or they could explicitly require some kind of license. I'd be shocked if Congress handled this productively _at all_; these issues are incredibly fraught in the best of times, Congress has been terrible for years at any kind of precise technical work, and the current Congress is uniquely incompetent.
I expect that if Congress accomplishes anything on this topic in the next couple of years, it will turn out to be awful for everyone.
Big Tech could also retaliate by making NYTimes conveniently disappear from search/news indexing
https://www.nytimes.com/2023/12/22/technology/apple-ai-news-...