Hacker News Reader

Comment by petra | Hacker News Reader

I don't haven ChatGPT but Gemini and Claude. But how do you make a language model think for 80 minutes ???

I have Gemini and ChatGPT and keep them on the highest thinking settings. ChatGPT will regularly think 40-60 minutes on the same problem that Gemini will think 10-15 minutes on. The quality of ChatGpt’s response is usually a little higher but not that much higher. My takeaway is Gemini is better at thinking faster, maybe has better more dedicated hardware behind it, and I use Gemini if I want a faster answer but ChatGPT I’d I want to push the quality of the answer a little higher.

I have the same experience, where Gemini thinks dramatically less than ChatGPT (or Claude), while achieving 90%-95% of the answer on it's first go. I'm surprised this isn't talked about more, because the difference is stark, usually around a factor of 5. This shows up in benchmarks too, where Gemini consistently uses many fewer tokens per solve.

So while ChatGPT produces a correct and/or thorough result after 10 minutes, Gemini got most of the way there in 2 minutes. The downside being you need to prompt again to get to the same level as ChatGPT, but you also can get ~5 prompts in the same amount of time.

I have claude to, but I use it the least because it limits so quickly. However its thinking time seems to be on par with ChatGPT

Probably because Gemini has access to Google's Knowledge Graph which has been around since 2012. One of the many major advantages Google has over other players that I also think is underdiscussed

More than that, Gemini has the entire knowledge graph team that probably got retooled to be an LLM data team now.

It has an “high effort” mode that makes it think really long

Ahhhh... you need ChatGPT pro at 100 bucks/month. Am I correct?

I believe so. With Pro you get “Thinking” with levels Light, Standard, Extended, and Heavy; and you also get the “Pro” model with levels “Standard” and “Extended”.

I don’t often go to Pro as it does take a while like you saw here, but I do often use Thinking Heavy for high quality answers. Idk why, but i just get consistently worse results with Gemini (Gemini pro), where it’s just much lazier, eg won’t do actual searches unless explicitly told.

In my experience, you can tell them "Don't stop working on this until complete" and they'll go for an hour or more.

Give it hard enough problems?

For that you would need Gemini Ultra

Reply on news.ycombinator.com