I don't need a model that can invent new mathematics. I need something that is fast, cheap, and consistent. Give me that and I can build and scale.
Where Gemini still wins is non-text input what Deepseek cannot do, yet, and Deepseek Flash has this thing of cheaper models where a failing tool call can derail your agent to a retry loop if you're not careful on instructions in the error message.
If they fix and make the tool calls to work better in non-optimal situations, it's much easier to switch from Gemini without a few weeks of evals and bugfixing.
- Medium for Gemini, high for Deepseek.
- Things like find information, then understand something about it, then send a slack message or email etc.
- Completion rates somewhere in 80-90%, Deepseek a bit better than Gemini
- Quality evaluated by Fable 5.1 and Astra 6.0 acting as a rubric judge.
Gemini quality would probably be better with high thinking level, but that would be 40% more expensive. And Deepseek is already third the price of Gemini.
From the large models Kimi K3 is definitely the one burning the smallest amount of tokens. Even if you pay for the fast version in Fireworks it's third of the price of Opus 5 for the same task.
All this really needs evals, the token prices tell nothing.
Yeah, that’s basically an industry-wide scam.
In any case old rules apply: if privacy is a concern don't share the data. I share all my work-related code because it's worthless, but I don't and would never share company business and process details, access to production/user data, etc.
Meanwhile I know of people connecting all the kind of MCPs for datadog/sentry/jira/concluce/production databases to their harnessess..lol.
Incredible good value and product they have built.
Works fantastic. Glad there is a more 'uncensored' thing to fall back to when the frontier folk are too sensitive.