Hacker News Reader

Comment by seydor | Hacker News Reader

I can already run gpt4xalpaca on my PC, a model that is not-bad-at-all and is completely uncensored (i.e. does things that chatGPT can't do). I think it's true that LLMs are racing to the bottom and will be even more once they can fit as a peripheral to every computer. whoever is investing in this to monopolize has not thought it through

View on HN

It’s astonishing to me that people seem to believe the llama models are “just as good” as the large models these companies are building, and most people are only using the 7B model, because that’s all their hardware can support.

…I mean, “not-bad-at-all” depends on your context. For doing mean real work (ie. not porn or spam) these tiny models suck.

Yup, even the refined ones with the “good training data”. They’re toys. Llama is a toy. The 7B model, specifically.

…and even if it weren’t, these companies can just take any open source model and host it on their APIs. You’ll notice that isn’t happening. That’s because most of the open models are orders of magnitude less useful than the closed source ones.

So, what do want, as an investor?

To be part of some gimp-like open source AI? Or spend millions and bet you can sell it B2B for crazy license fees?

…because, I’m telling you right now; these open source models, do not cut it for B2B use cases, even if you ignore the license issues.

You know what I believe is also a toy model? chatGPT Turbo, you can tell by the speed of generation. And it works quite well, so small size is not an impediment. I expect there will be an open model on the level of chatGPT by the end of the year because suddenly there are lots of interested parties and investors.

Eventually there will be a good enough model for most personal uses, our personal AI OS. When that happens there is a big chance advertising is going to be in a rough spot - personal agents can filter out anything from ads to spam and malware. Google better find another revenue source soon.

But OpenAI and other high-end LLM providers have a problem - the better these open source models become, the more market they cut underneath them. Everything open source models can do becomes "free". The best example is Dall-E vs Stable Diffusion. By the next year they will only be able to sell GPT4 and 5. AI will become a commodity soon, OpenAI won't be able to gate-keep for too long. Prices will hit rock bottom.

> I expect there will be an open model on the level of chatGPT by the end of the year because suddenly there are lots of interested parties and investors.

I really don't think you understand just how absurdly high the cost is to train models of this size (which we still don't know for sure anyways). I struggle to see what entity could afford to do this and release it as no cost. That doesn't even touch on the fact that even with unlimited money, OpenAI is still quite far ahead.

Still cheaper than a plane, a ship or a power plant, and there are thousands of those.

And how many are given away for free?

I think you're conflating speed of inference/generation with optimization. gpt-3.5-turbo does not fit on a single GPU unlike the "toy" models.

I think that Alpaca 30 billion is pretty competitive with ChatGPT except on coding tasks. What benchmarks are you using to make your determination about suitability for B2B?

gpt4xalpaca is 13B

7? 13? Who cares? It’s an order of magnitude smaller than the GPT models. It’s a toy.

This is a repeat of the early GPU era.

It's not the software or hardware that will "win" the race, it's who delivers the packaged end user capability (or centralizes and grabs most of the value along the chain).

And end user capability is comprised of hardware + software + connectivity + standardized APIs for building software on top + integration into existing systems.

If I were Nvidia, I'd be smiling. They've been here before.

Nvidia: just as the sun starts setting on crypto mining, the foundation model boom begins. And in the background of it all, gaming grows without end.

If you've got a choice, sail your ship on a rising tide! And if you can spread the risk over multiple rising tides, so much the better!

My dad told me a quip once: "It's amazing how much luckier well prepared people are."

> I can already run gpt4xalpaca on my PC

You can also run your stack on a single VPS instead of cloud, gimp instead of photoshop, open street maps instead of Google maps, etc.

There will always be companies who can benefit from a technology, but want it as a service. In addition, there will be a lot fine-tuning of LLMs for the the specific use case. It looks like OpenAI is focusing a lot on incorporating feedback into their product. That’s something you won’t get with open-source models.

Imagine you're a tech company that pays software engineers $200K/year. There is a free open-source coding model that can double their productivity, but a commercial solution yields a 2.1x productivity improvement for $5000 annually per developer. Which do you pick?

Not sure if parent had a certain answer in mind, but my answer is OSS because (1) I can try it out whenever I want, and (2) I don't have the vexing experience of convincing the employer to purchase it.

That’s the endless «build vs buy” argument. And countless businesses are buying.

I don't this it's the same thing, at least for me.

In the GP's scenario, I wouldn't be building either piece of software.

The existence of the models is making programmers cheaper rather than the reverse.

But i think it is underestimated how important it is for the model to be uncensored. ChatGPT is currently not very useful beyond making fluffy posts. As a public model, they won't be able to sell it for e.g. medical applications because it will have to be perfect to pass regulators. It cannot give finance advice. Censorship for once is proving to be a liability for a tech company.

In-house models OTOH can already do that, and they can be retrained with additional corpus or whatever. And it's not even like they require very expensive hardware.

I find your argument persuasive, companies should spend extra for the significant productivity gain. But then again from experience most companies don’t give you the best tools the market hast to offer..

Yeah but with very simple tasks with the 2k tokens limit. Let alone the fact that it can't access the internet, or have more powerful extensions (say Wolfram).

Alpaca is the Napster or LLMs

Reply on news.ycombinator.com