Hacker News Reader

Comment by hrpnk | Hacker News Reader

I love how there is a "Please do not discontinue gemini-2.0-flash[-lite], 2.5 is NOT an equivalent" from Feb 20th. Getting too attached to models is a smell.

View on HN

It's not a smell. Why should these developers rebuild a core piece of their stack every few months. Switching out a model requires a new round of testing and validation when we should be able to rely on a piece of software the behave the same way since the last time we touched it.

> Why should these developers rebuild a core piece of their stack every few months?

That's what they signed up for when established a hard dependency on an subscription online-only LLM model.

If they were on open weights, at some point the provider would deprecate it, probably with worse notice

And self-hosting would probably have been more expensive unless they had massive volume: Deepseek V3.x would have been the comparable open weight model for the performance and isn't that cost effective until hosted across multiple nodes with large batch sizes

Sounds like something you'd need to evaluate before adopting then.

This is how development looks like for many years now, constant rewrite on the horizon. I think LLM development hype surpassed Blockchain and JS frameworks craze of decade ago.

It's kind of the same problem with cloud in general (though that moves much slower).

If you want to be sure to be in control, then host it yourself

Its almost a given considering how fast this field moves. Also, what kind of workflow structure would someone have that a single specific model is the only one that would perform acceptably?

Programs are tightly shaped to their dependencies and software frequently breaks when bumping a dependency versions. That's why we have package managers that are carefully built around lock files. Models should be thought of like a dependency, not just an endpoint to a workflow.

These are AI models designed to be non deterministic. If upgrading from GPT 5.5 > 5.6 breaks your entire workflow, you shouldn't be using AI to begin with

> Why should these developers rebuild a core piece of their stack every few months. Switching out a model requires a new round of testing and validation when we should be able to rely on a piece of software the behave the same way since the last time we touched it.

Because fuck you, that's way. /s

There's a massive amount of contempt from a lot of software businesses for their customers, and it's just getting worse with LLM models.

In the post the issue is performance. Are you saying that getting too attached to performance is a smell? That sounds very odd.

It's not because a model performs better in some applications (often by fine-tuning to get better scores at specific tests) that it is better across the board or that we have to believe the company releasing the model with a high number 3 > 2 so that it is commonly accepted as better.

Pushing the reasonnning further: f you need an Opus level performance then not accepting GPT 3 isn't a smell.

The smell is about folks being attached to the same model because they're used to it. Just like any UI change in an app that one uses everyday is annoying.

What's often missing is the actual engineering. If you have evals for your use case, you use these to adjust to the new model. DSPy has great prompt engineering constructs it ships with.

The smell is all about vibing, where a model feels better because the structure of the answers is more familiar to a person, instead of engineering where given constraints and input/outputs one is using/bending the system to fulfil the requirements.

the 1.5 and 2.0 flash models were absolute beasts. They were very cheap, and _very_ fast. We contemplated moving some of our fine tuned workloads to them because we would have gotten very substantial total latency reductions for our workloads.

However, they are aggressively deprecating them (OpenAI is as well), and replacing with newer models. These newer models are all reasoning models, and importantly, only bear the flash name. They are not fast. And they are very expensive!

I test workloads with multiple closed and at least one open model now. Good to have a backup on 503s or credits run out.

have you experimented at all with the deepseek flash models?

That's like saying 'getting attached to locked dependencies for your app is a smell'.

But this could be framed as 'getting attached to an API revision when a new one is available'...

I can see it both ways, tbh.

We have benchmarks for our use cases, and every generation after Gemini 2.0 Flash has been a grim hit on price/performance. Costs have gone up, throughput has gone down, and performance has improved very slightly (and regressed on a few things).

I built some BigQuery workflows on 2.0 and 2.5 flash lite that are something like 6x more expensive with 3.1 flash lite.

I tried 3 flash for months and it didn’t work using Googles own vertexai integration because it’s been in preview mode for months.

Not wanting to pay significantly more and do a bunch of rework isn’t a smell.

They left a large gap in their new pricing vs the prior generation, and if you had a working use case that sucks. The model is >99% reliable for my use case so there’s nothing to gain from a smarter model.

Reply on news.ycombinator.com