Hacker News Reader

Comment by astrange | Hacker News Reader

The problem is forbidding it won’t do much.

- It appears people can train AIs from scratch or at least fine-tune them at home.

- Even if your art isn’t in “the training set”, that does not prevent the AI from learning its style. (Someone can decode it to CLIP embeddings. It could have a really good text model trained on vivid art museum descriptions of your art.)

- The ability of an image model to generate your art means it could also be trained in reverse to recognize it, producing a caption model, which would give vision to the blind. And surely you’d feel bad about that.

View on HN

If you forbid it, the investment in developing those models disappears. They will be stuck at ~what we have now at best.

You can also require cloud providers to enforce a ban on training (and deploying) such models, it's doable. Good luck training it in your basement, it will probably take you a decade.

If this is banned, it will become a lot like piracy - yes, it's available, no, most people (at least in the West) don't do it, practically no businesses do it.

Training these models is much much cheaper than you think it is, and there’s good data for it already.

Either use a CC0 set like Wikimedia/Flickr and throw in some dead artists like Brueghel, or train on data from a country we don’t respect the IP of. Lots of Taobao product photos out there. It’s enough.

You are getting ridiculous now. Training this on "Taobao product photos" will lead to a useless model that is unable to produce practically all of the "cool" demos posted here in the last week.

A few months ago this task was virtually impossible. Then it was possible, but extremely expensive and pay-walled behind the "Open"AI' website.

As of about a week ago this tech runs on consumer GPUs. The weights have been downloaded 100s of thousands of times, and fine-tuning / modifying is possible.

Training from scratch is about $500k still, but it will only get cheaper and easier.

This doesn't contradict anything I have written. The average technical user will be unable to train this exact model (not to mention the supposed future more powerful ones) in their basement in this decade.

That just feels like such a pessimistic forecast to me. Of course, the current trajectory of improvements in model efficiency and better commercial GPUs / ML-accelerators may hit a wall.

But I would not be surprised if this was trainable on a commercial GPU at home within that time. But I think another important trend that we are seeing is that you don't need to train these models from scratch.

Open-source "foundation models" means that you can usually get away with the much easier task of fine-tuning, as to not throw away / re-learn everything that these large models have already fit.

Edit: I initially said 2-5 years, but on more reflection this does seem optimistic (for training from scratch).

If things go that way, the 'legit' models will continue dev, just using licensed content (along with public domain works). It will be more expensive for the end user, but that cost will shrink over time for general work. Tools that mimic working artist though might not be available (or will expensive). This all seems pretty ideal, so the pessimist in me guesses it's fairly unlikely.

I am not sure it will be possible to get enough training data that way.

I don't know enough about diffusion models but if LLMs (of current size) have to use only public domain, they will be undertrained and we will see significant degradation in performance. Not to mention that Codex will be effectively dead.

Reply on news.ycombinator.com