It’s so annoying that everyone just points to the Artificial Analysis index (or even worse, Epoch AI, where part of the score is how good the AI is at chess) as a proxy for “how good” the model is.
Its 200-300 TPS. GPT is at 50-60. And its like 5% worse? Yeah that is a good tradeoff.
Are you watching/waiting on your agents? I care zero about t/sec, quality is far more important that quantity or latency, but they run in the background and I check in from time to time
Most people talking about Gemini quality do so by experience.
sounds vibes and bias laden
what's your solution? force people to use gemini until they like it? or hire them as googlers?
my approach is to not use closed weight models, I'm not trying to solve for how to get people to use gemini, quite the opposite actually