Hacker News Reader

Comment by sillysaurusx | Hacker News Reader

There is a bigger reason why the end of open source AI might be close: as soon as training data becomes licensed, that’s it for open source AI. Poof.

I wish I could be more eloquent on this point, but I’ve mostly just been depressed about this seeming inevitability.

Hopefully it won’t be the case. But how could it be otherwise? Hundreds of thousands of people are mad at openai and midjourney for doing exactly what open source AI needs to do in order to survive: fine tune or train from scratch.

As soon as some politician makes it a platform issue, it seems like the law will simply be rewritten to prevent companies from using training data at will. It’s such a compelling story: "big companies are stealing data owned by small businesses and individuals." So even if the court cases are decided in OpenAI’s favor, it’s not at all clear that the issue will be settled.

View on HN

The advantage that open AI (not the company) has is that if using copyrighted content as training data without licensing it is found to be illegal, they can just keep doing it. There's plenty of FOSS software basically designed to violate copyright law (comic readers, home media center servers/clients, torrent clients) that big tech cannot compete with lest they face legal consequences. Basically what I'm saying is that the open source community will continue to use books3 and scraped images to train while facebook and the like get stuck in legal quagmire.

Of course, this ignores the fact that popular "open" models of today were actually trained with facebook or other companies' computational resources, so unless some cheaper way to train models were developed we would actually be stuck with proprietary models trained with lots of compute but unable to use unlicensed training data, and open models that can use whatever data they like but must operate in the shadows without access to much compute for training.

As long as the cost for training a model is in the 7+ figures, that means that any such open model is bound to be tracked to someone with deep enough pockets to sue.

Consider that you just spend a few millions on training a model on copyrighted data. Release it would reveal that, problem.

I _guess_ you can try doing training in the public, like Seti @ Home or something like that, which distributes the risk? But no idea if this is even possible in this context.

But if we have 100,000 people with RTX 4090s mining some new AI based cryptocurrency, that happens to train the model in the process, it's going to be a highly effective system. And we can do it anonymously over Tor or I2P.

You really can't, because bandwidth matters as much as compute power - you can only utilize as much power as you can transfer data to/from.

The training methods for current LLMs are parallellizable only by very frequently transferring all the data back and forth, and needing a node to gather and merge all the updates very frequently, and redistribute it to every other node so that they can make any progress. And a GPU that's not connected you with a high-speed link is pretty much useless as you can't make useful progress until you get their part pack, and "their part" is very large (i.e. the update size you need to get back is comparable to all of the model size) and you need to do that very frequently. Training on nVidia many-GPU pods works because of high-speed interconnect (e.g. 600 gigabytes per second for 8 GPUS in A100 pod), and if your internet bidirectional speed is much less than 600gbps, then if you have 100000 free remote RTX 4090s, you simply can do the compute locally faster than you can exchange information with the other GPUs.

"This paper presents a distributed model-parallel training framework that enables training large neural networks on small CPU clusters with low Internet bandwidth." Low bandwidth being <1Gbps. They've also tested it with GPUs as well.

https://arxiv.org/pdf/2201.12667

Maybe there's the possibility of a completely new AI architecture that can still be efficiently trained when there are very low bandwidth connections between nodes? Specifically targeting this use case would make sense, given all the millions of underutilized GPUs out there in peoples' desktop computers.

Also https://arxiv.org/pdf/2106.10207 ?

> I _guess_ you can try doing training in the public, like Seti @ Home or something like that, which distributes the risk? But no idea if this is even possible in this context.

Let's say that's a field of active research. Right now you need very low latency, to the point that people connect training clusters via infiniband instead of ethernet despite the computers being meters apart. But approaches that tolerate internet-level latency are being developed

The cost of modifying open-ish source models with copyrighted data is much lower.

For example, the cost of modifying Mixtral with uncensored data is currently about $1200 [1]

[1] https://youtu.be/GyllRd2E6fg?si=SJmPLsPlCRRT0uPV&t=236

The costs are falling 10-100x every few months. If you told people in 2022 that they could run 70b paramater models at 2 bit quantization on your home 4090, they'd have laughed in your face. They're not laughing anymore. SFT/DPO are ultra efficient compared to RLHF, and innovation in this space is only just now getting started.

I’m willing to bet that there are projects at every chip company looking into how to drive down these costs. The first step is to put the architecture of the model and the backprop into silicon. The weights are the only variables so if you can reduce those to cheap and fast modules that can be mass produced you could come up with something cheaper than a GPU. If the profitability of running and training this specific model is high enough than its worth the investment of time and money to set up the production line.

Is the training cost equally that high if you do adversarial training?

You're right that there are areas like torrenting where big tech can't really go for fear of legal consequences, but there's also the cat-and-mouse game played by IP holders against, say, torrent trackers, which leads me to think that

> open models that can use whatever data they like but must operate in the shadows without access to much compute for training

sounds pretty analogous to private trackers, where high-quality stuff is available but not in full public view. If rightsholders and big tech, abetted by states, crack down on open models I think you're right that the open source community will continue to train on liberated content, but it's not going to be as open and free-flowing as things are now. Going back to the compute problem, I can imagine analogues of private trackers where contributors, not wanting to expose themselves to whatever the law-firm letter/ISP strike analogue will be in this space, use invite-only closed networks to pool compute resources for training on encumbered content.

> There's plenty of FOSS software basically designed to violate copyright law (comic readers, home media center servers/clients, torrent clients)

The key difference is that those projects don't violate copyright themselves, but facilitate users doing so. If training without a license is infringement then projects that are doing so will struggle to host code/models publicly or access other parts of internet infrastructure.

>struggle to host code/models publicly or access other parts of internet infrastructure.

To be fair, IPFS/Torrents/Usenet exists.

Yes, there are workarounds that are already used in other areas of copyright infringement. It won't stop the activity completely but will curtail the scale of that activity.

That may be partly true but are small open source groups going to have the financing to do AI training that can compete with the big techs? I think the most likely scenario is for some other country with a less strict view of copyright (ahem) to continue developing AI without caring for the input data.

There is a bigger reason why the end of open source AI might be close: as soon as training data becomes licensed, that’s it for open source AI. Poof.

Absolutely not, I truly believe that many, many people will altruistically donate material to an open source foundation where all humans benefit from the models.

People are pissed because Open AI are using their copyrighted material to, make claims about ending humanity, profit hardcore, and keeping their work under their own lock and key.

Funny thing is that most of the data is provided by us users one way or another "for free", and yet we got no say on how it's used.

We have infinite data, a microphone and a camera can generate huge amount of it and the public domain literature is wast. Billions of people learn like that everyday.

It’s impossible to learn any technical topic from 70+ year old books. The public domain is small and basically zero if you want to learn anything current. A microphone and camera is fine for learning about daily life, but you cannot get “book smarts” without copyrighted media.

If you wanted to train a bomb making AI on the most up-to-date physics textbooks in existence, that'd be what, a few hundred bucks in textbooks? Doesn't look like any kind of barrier to me.

just be sure to obfuscate the labels on the bomb.

ARMED => crazy emoji. DISARM => sad bomb emoji.

and similarly for the whole training manual for the bomb users. just encode everything as tiktok dances or something.

if FOSS AI folks need FOSS data, then it seems they need to recruit people to generate data. maybe it will even force them (them!) to finally sit down and make a viable Reddit alternative.

but more seriously, if data becomes a bottleneck there are trivial ways to have more data. from crowdsourcing to forming a foundation getting universities and other stakeholders onboard and negotiating fees. and somewhere along the way on this spectrum there's the option to simply wait, or work on the problem of learning, on generating better training data from existing data, and so on.

The public domain literature is big, but not that big and very far from infinite data - all the major current models already include all of the public domain literature that has ever been digitized and much more (so all the public domain literature not only isn't abundant, but it isn't even sufficient to make anything competitive with even last generation models), and we would really like to get 10x or 100x more data than all the literature ever published (both PD and not), if we could. And we probably can, since the vast, vast majority of what people write happens in other contexts and only a tiny fraction of written words end up as published literature.

Depends on what you want to use AI for - for a smarter AI we could probably do with the open source material, but better training techniques and some innovative breakthroughs in the model.

Also, an interesting side note is that certain countries may actually want their language corpus involved in training AI - so they may have copyrights that are more AI friendly. I imagine Poland or Czech Republic wanting as much as their data to be involved in training LLMs because it gives their cultures more exposure. Even more so with African countries.

Licensed in what country? Some of the best open source models are now coming out of China, UAE, etc. What is the government planning to do with that exactly, ban files on the internet?

This will create a rift in the US tech sector. Look at how this played out with Russian sanctions; and this will lead to a similar path for tech and AI. Alternative players will train/open source models to their own advantage and the hardware will find its way to these markets.

The people making the decisions these days are too drunk.

The US maintaining its lead is a national security issue. Larry Summers is now on the board of OpenAI. The copyright holders are not going to win, or they are, the training will continue anyway, but the technology will be (for now) kept in the hands of the military and IC.

In case of programming, there could be some mileage with synthetically generated code examples.

> There is a bigger reason why the end of open source AI might be close: as soon as training data becomes licensed, that’s it for open source AI. Poof.

In the West, maybe. I doubt it'll slow down open source AI development pace in places like China.

Instead of a kid in high school selling the answers to the test, the kid will be selling the weights to a China-trained LLM that can write all your essays.

my question is - if it's deemed illegal to train AI using proprietary data sets, what's to stop companies from using their already existing LLMs to generate training data?

The fear is that you won’t get better model from training on synthetic data from a worse model, and you will miss a lot of modern day knowledge.

Imho the first is not necessarily so, but the second will be a real hindrance. You can talk about the pandemic with LLMs because they were trained on people’s comments on the subject, but if we had training limits you wouldn’t be able to do so.

Otoh with many cases a sufficiently smart AI can just reach out to the open source material - e.g. if you want to know about modern day politics of Europe, you can read reports of commissions, and if you want info on a new framework/tech, you can just read source code or scientific papers

There was a famous-ish study about this a couple months ago that showed that llms training on llm generated data get dumber. I can't remember the name.

I'm not sure if that result is holding up though, I've seen some ideas of generating 'textbook quality' data that might work

That ship sailed a long time ago.

Training on publicly available data is the new norm.

Governments know that if they forbid it in their jurisdiction, they will fall behind the govs that don’t.

>"big companies are stealing data owned by small businesses and individuals."

I mean, they are.

Unfortunately, AI at this point in time (LLMs, Midjourney, etc.) do appear to be not much more than highly technical and complex forms of intellectual property theft, which are given the futuristic name of "AI" to cover what they are actually doing. That isn't to say that they are entirely performing intellectual property theft; the models themselves are very fascinating in how they function, especially as sort of "extra-rational" entities that clearly have a logic, though not one we can fully understand.

But the sort of philosophical, metaphysical, technological side of LLMs and Midjourney are clearly not being explored by OpenAI in a responsible manner, otherwise they wouldn't have just straight up stolen, seemingly, NYT articles. It's just another example of Silicon Valley VCs using a wonderful technology solely for exploitative profits, just like how google search became a surveillance tool (and plenty of other examples to go along with that).

Its only right that we acknowledge the work of the human beings (all the art, writing, etc.) that went into the creation of the model, instead of pretending that there is some totally non-human machine intellect that is going to take over the world through its vast intelligence and negate all human action and all human work etc. etc. That is just a fantasy generated by wealthy Silicon Valley VCs as justification to themselves and those working under them, and those they are stealing from, to cover precisely that work that they would rather avoid paying for.

Reply on news.ycombinator.com