Hacker News Reader

Comment by valine | Hacker News Reader

AI models with obvious, broad, real world applications always seem get reproduced in public. Nvidia’s result is obviously great, but it’s still a long way from being useful. It reminds me of image generation models maybe 4 years ago.

We need a killer application for video generation models first, then I’m sure someone will throw a $100k at training an open source version.

View on HN

I think deep down, everyone already knows what the "killer application" for open source AI video generation is going to be.

Yeah, just visit civitai and see what's popular there.

Reducing human trafficking by generating artificial content ?

Not a thing.

I am going to guess: Making "dog-nature-like" humanoids that aim to please lonely people, who are nicer to be around than people, easier than real relationships.

Don't we have enough of that already?

Capitalism doesn't ask that question.

> AI models with obvious, broad, real world applications always seem get reproduced in public.

Really? GPT-3 was released almost 3 years ago. Where is the public reproduction?

And don't say LLaMA. I've used it and it isn't even close to GPT-3, nevermind GPT-4.

The current generation of GPT-3, which started with text-davinci-003, was actually released on November 2022, not quite 3 years ago. I'm not even sure the model that was released 3 years ago is still available to test, but it was much less impressive than more recent models - I wouldn't be surprised if LLaMa were actually better.

The model trained 3 years ago was only trained on 300B tokens, heavily undertrained (in terms of the Chinchilla scale), that's why LLaMa models can easily beat it on most benchmarks (they were trained on 1/1.4T tokens). About the current GPT-3.5 models, who knows, OpenAI is not very open about it.

Which Llama model have you played with? Unless it’s Llama 65B with 16f precision it’s not a fair comparison to gpt3.

The tragedy of the commons is at play here. We could get amazing ML models rivaling the best if people interested could pool together money for a $100k or $1 million training run. But there doesn't seem to be the willingness, so we humbly rely on the benevolence of companies like Stability and Meta to release models to the public.

As someone not very familiar with the field, what is wrong with this comment? Is the scale off?

Kickstarter is into the tens of millions these days [1]. I would assume some number of millions might be possible, if the right names were behind it.

[1] https://www.kickstarter.com/projects/dragonsteel/surprise-fo...

Kickstarters like the one you linked don't suffer from the tragedy of the commons because people are essentially just pre-paying for a product. With funding an open source ML model, there's little incentive to not be a free-rider.

I doubt there would be much funding for a kickstarter that didn't promise to be open source.

> people are essentially just pre-paying for a product

I think this is what the 4 nines of the user space wants. A pre trained, open source, model to work with.

Reply on news.ycombinator.com