Hacker News Reader

Comment by nsoonhui | Hacker News Reader

Your benchmark has Gemini 3.5 Flash as the best model, which doesn't compute for me

It is on top for many benchmarks, only not the coding/agentic ones.

Still one of the most intelligent models overall, most likely to get any question you ask correctly (without tools).

Not in my experience, it tends to pick up subtle orientations given in a question (like "which is better, A and B?" and in the context you add you list a few things for A and B) and will absolutely run with them even if they're not true. Has been an issue with Gemini models since at least 3.0. Maybe that makes them great roleplaying models, but for factual information they just run with the slightest hint in one direction or another and never really push back objectively.

This guy had a terrible broken benchmark that gets hawked every release, and I wish HN would ban accounts that essentially exist to hawk a personally owned site, especially such a bad one.

I get similar results in my own tests. And Gemini 3.1 Pro is consistently on top of my ratings. Not everyone is coding monkey, I prefer staying a programmer.

They're referencing Gemini 3.5 Flash being the top model, you must not be great with detail.

And no (strong) programmer would jump to assuming other people are coding monkeys just because they disagree on what a strong LLM is: that's the kind of thinking reserved for the glorified coding monkeys who wasted their life getting better at writing CRUD apps and are now upset that someone's tooling is dropping the already very low bar there.

If you were right, the karma system would largely take care of this. It really sounds like this is more of your personal view

Karma systems are never perfect, and most people will not assume this is a pattern.

(ie. won't feel the need to downvote them just for having yet another crappy AI benchmark)

I only recognize it because I build a product that leaves me looking for information on every major release... and every major release a new crop of folks reply confused about the anomalies on top of anomalies that they're seeing, and they slowly learn this person is just way more unserious than the dogged distribution would imply.

Reply on news.ycombinator.com