Hacker News Reader

Comment by hackpert | Hacker News Reader

Wow I had hoped for a more productive discussion than these 1-1 comparisons of Bard vs ChatGPT that I'm seeing everywhere. The model deployed with this version of Bard is clearly a smaller model than the biggest LaMDA/PaLM models Google has been working on for ages. Which, according to their publications, show unprecedented results on _proof writing_ of all things (see Minerva). While their strategic decisions may be questionable (or they're just trying to quantize the model for mass deployment without burning billions per month in compute costs), its almost silly to question Google's ability to build useful LLMs.

View on HN

At the moment unless we get more information about what metric you're supposed to evaluate it on, you could probably simplify the headline to just "Bard is much worse than chatgpt" without any loss of accuracy.

It's not really realistic to expect people to give Google credit for these amazing models they have published results about but haven't let people play with - they have given people Bard and people are evaluating it based on the criteria most obvious to them - a comparison to a very similar product that has just been released.

They knew the war they were entering, they knew their enemies, they knew how they'd get evaluated and still decided to get this model out in its current state, leading to the conclusion: Yes, this is really the best they can do and it's much worse than the state of the art.

In any case, it's a massive marketing blunder, the public opinion formed within the last hours was overwhelmingly "Bard sucks compared to ChatGPT."

>Yes, this is really the best they can do and it's much worse than the state of the art.

This is the best they can do under pressure.

ChatGPT surprised the world with how good it was, then Google scrambled to get something out quick.

A project like this is a massive undertaking, the first mover has the advantage that they can calmly refine their model until they find it presentable.

The question is, is what Google is delivering good for the timeframe since OpenGPT exploded in popularity enough for Google leadership to take note. Since that moment, realistically, is when they put pressure on their devs to push something out the door.

I think we'll see a better iteration soon. Not only from Google, but from other competitors.

> its almost silly to question Google's ability to build useful LLMs.

Unless they release a model one can "use" and verify their claims it's literally silly to make this statement.

There are useful open source LLMs. Or are you questioning their ability to configure; make install?

> its almost silly to question Google's ability to build useful LLMs.

It's almost silly to presume anything without proofs. People are judging Google based on what Google has shown.

It seems like they don't want to be the best, just good and cheap enough that they don't lose users therefore ad revenue.

They behave like Yahoo when Google took over.

Even the best Google models seem to be lagging for reasoning tasks vs OpenAI ones at the moment - see the graphs at https://github.com/suzgunmirac/BIG-Bench-Hard

Everyone’s gonna have bigger models. Where are their useful models?

Thats like saying car brand X is more reliable than car brand Y, because brand X won formula one.

you can always go back to reddit

Reply on news.ycombinator.com