Hacker News Reader

Comment by neel8986 | Hacker News Reader

At this point OpenAI should definitely think of some sort of partership with AWS and GCP. Otherwise they will be just one of the few models

View on HN

I personally didn't realize how fast other models would catch up to OpenAI.

There is a whole set of models now (and some like Meta are purposely trying to undermine OpenAI competitive advantage via open source models) and they are relatively interchangeable with nearly no lock-in.

OpenAI's main advantage is being first to market and having the strongest model (GPT 4), and maybe they can continue to run ahead faster than everyone else, but pure technical leadership is hard to maintain, especially with so many competitors entering.

> I personally didn't realize how fast other models would catch up to OpenAI.

They haven't though. Gemini is vaporware and other models are not as good as GPT-4.

Mistral Large is quite close to GPT 4 per the stats from today:

https://mistral.ai/news/mistral-large/

Now once OpenAI launches GPT 5 well I am sure other models won't look so good, but right now these other models are approaching GPT 4 capabilities.

I'm pretty bearish on GPT 5 being better than 4. With how neutered 4 has gotten over time, I'd be surprised if GPT 5 is able to perform better with all the same restrictions that GPT 4 has. GPT 4 is less and less willing to actually accomplish a task for you than it is to tell you how you can do it. It looks more and more like Markov chains every day.

Did you mean bearish?

Yes I did, thanks.

I find gemini advanced better than gpt4. it is my go to option now.

For now. As others have said, there is no technological “moat” in this business that could prevent others from catching up.

Perhaps the best way for Open AI is to become THE established AI services company. AWS is still the leader in cloud computing space, and only has Azure competing, despite the fact that other big companies are also technologically capable of building similar products.

> AWS is still the leader in cloud computing space, and only has Azure competing, despite the fact that other big companies are also technologically capable of building similar products.

What happened to GCP? I personally switched away because of the bad experiences.. but is that happening in scale as well. I see it barely mentioned these days.

GCP is suffering from bad sales strategy and lack of existing enterprise relationships.

Their main advantage for now is their super clean API. Open source alternative are already on par with GPT-3.5 and 4 capabilities, they just don't have as good a package but that could change rather quickly too.

Mistral's API was designed to be practically interchangeable with the OpenAI API.

What open source alternative is on par with GPT4?

Is that true? I was running Llamas on my laptop a few days ago, and it was giving measurably worse results than ChatGPT. I think it was the uncensored 13B model, but if you got something that's on par with ChatGPT that I can run on my own hardware I'm pretty interested.

13B models probably cannot directly compare with ChatGPT 4 which maybe +1T parameters or a 5 way MoE of 200B each - or something like that. So you can not likely run a model competitive with ChatGPT locally in the near term.

I have a server with a bunch of PCIe slots and like 4 Nvidia GPUs with 24GB of RAM each. What's the best model I can realistically run?

Here are some scorecards:

https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...

https://paperswithcode.com/sota/sentence-completion-on-hella...

https://paperswithcode.com/sota/common-sense-reasoning-on-wi...

https://paperswithcode.com/sota/common-sense-reasoning-on-ar...

https://paperswithcode.com/sota/common-sense-reasoning-on-co...

> Open source alternative are already on par with GPT-3.5 and 4 capabilities

I'm not sure if this is true. With GPT-4, I can successfully ask questions in Japanese and receive responses in (mostly natural) Japanese. I have also found GPT-4 capable of understanding the semantics of prompts with Japanese and English phrases interleaved.

Out of curiosity, I tried doing the same with local models like Mistral 7B and I could never get the model to emit anything other than English. Maybe it's a difference in training data, but even then, GPT-4 has an allegedly small set of training data for non-European languages.

Sure, but it is somewhat disheartening to see GPT 4 still being the king by a clear margin after a full year, especially since it's been nerfed continuously for speed and cost effectiveness.

Their problem is that they had to agree on an exclusive relationship with azure in order to secure funding from Microsoft

Honest question from someone who never touched one of those models, is OpenAI anything else than the first and most hyped model developer?

IME nothing is as good out of the box as gpt4 for many tasks.

Is gpt4 as good in non-English uses? It's not clear to me that it would be particularly important or advantageous, but does Mistral being based in Europe and polyglot first make it interesting vs. gpt4 in some dimension?

I guess it might depend on language, but as a Spanish speaker who sometimes uses LLMs in Spanish, I'd say the gap between GPT-4 and most of the competition (Mistral included) is actually larger in Spanish than in English.

In my experience it’s not such a simple question. If you want to be able to speak in nuanced non-English and have it pick up on the intricacies, or have it respond in rich correct non-English, then it’s not the best model (Cohere recently released an aya model that I would recommend checking out if this is your use case).

If you want to be able to give basic commands and have the model reason about the logic behind your commands, gpt 4 is still the best, even in minority languages.

gpt4 is going to better than other models in every language except maybe chinese

Depends on the task. Gpt 4 isn't necessarily better at translation than Claude. I'm aware of no benchmarks on this.

From what I heard Gemini might be better at translation tasks than GPT4.

It's the best multilingual model out there and it's not even close.

Especially in terms of open models Mistral's are the most multilingual but outside a few handpicked ones the level of proficiency is just too poor for any real usage.

I don't disagree with you, but an open source model fine tuned for your use case, and embedded with your data is probably going to be way better at many companies uses cases than GPT4 is.

Yes, GPT-4 still rules, downside is it's expensive and relatively slow.

I'd say you should compare the models for your use case. Which is better depends on how much you're willing to pay, what kind of problems you need help with, speed, ease of use.

Reply on news.ycombinator.com