Hacker News Reader

Comment by MuffinFlavored | Hacker News Reader

If this is 13B and we can assume GPT-4 is 100B+, is this even in the same ballpark as useful?

Being fast at generating "something" (spitting out tokens) is one thing but... if those tokens aren't worth much (aka if the quality of the "answers" are weak), what is it really worth?

View on HN

For me the big difference is that I can run llama models locally which is a different use-case entirely compared to GPT-4. Even if the weights were available, I just don't have the hardware to run something with the parameter counts of GPT-3.5+

> what is it really worth?

For some, nothing. For some, a lot.

If you’re looking to create a ChatGPT replacement, this ain’t it.

If you’re trying to learn about LLMs, how they work, capabilities vs deficiencies, private data use, etc., these are invaluable to get your hands dirty in a private environment.

I don't get what running/researching an LLM that is really bad at hallucinating is worth.

Don’t think production code, think MVP that may allow you to prove certain hypothesis and/or demonstrate need.

That may/will allow you to raise funds which will allow you to actually pay for OpenAI’s API or maybe train your own LLM or whatever.

Again, if you think in terms of MVP, these become extremely valuable.

GPT3.5 was 175B parameters and GPT4 is larger (nobody knows how larger) and it blows GPT3.5 out of the water.

So, GPT4 is, well, way more than 100B+.

And yes, I share your opinion

The ability to run it on your own private data and train/distill it from the larger models is very useful (see Orca for example: https://arxiv.org/abs/2306.02707).

Most of the valuable data in the world is private data unsuitable to send to GPT4 and other proprietary models.

> is very useful

How? Is there not extreme correlation between "13B parameters (low/not a lot) most likely means lots of hallucinations"

What’s better a 130bn model or 10 13bn models working together.

The GPT4 cognitive leaps and breadth of knowledge seem impossible to replicate with any number of 13B models.

13B x10 might be the only feasible business model for your company, but that alone doesn't make it a superior option. Can you share more details on how the models work together?

There is no information about the size of GPT-4; it could very well be smaller than GPT-3.5.

Sam has said multiple times "the age of large models is over" https://www.wired.com/story/openai-ceo-sam-altman-the-age-of...

Either way, the LLaMa models are quite powerful and now that we have an open, commercially usable version in 3, 7, and 13b sizes is huge. The 65b model I'm assuming is coming soon as well.

I used chatgpt4 daily.

I tried using some of these 13b models and was completely underwhelmed when compared gpt4.

I think some of these 13B models may only be pretrained and may not have benefited from RLHF. So it’s not clear how much of the difference is due to parameter count.

There is now published research demonstrating that small models (LLaMA 7B) surpass GPT-4’s performance when fine-tuned on specific tasks.

> There is now published research demonstrating that small models (LLaMA 7B) surpass GPT-4’s performance when fine-tuned on specific tasks.

Got any examples? I don't understand how it can "hallucinate like crazy" for some but not for all

1. Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Tasks https://huggingface.co/papers/2305.14201

2. Gorilla: Large Language Model Connected with Massive APIs https://arxiv.org/abs/2305.15334

Reply on news.ycombinator.com