Sure… but which ones? How can you know ahead of time?
I just did a “simple” upgrade project where both me and the AI kept tripping over dead code, subtle typos, and difficult-to-trace live versus dead code.
Many times I used “Medium” thinking I got bitten, but not every time, and I couldn’t predict when.
So “Extra high” it was, for the entire project.
Far fewer nasty surprises!
For example, I have software that summarizes articles and classifies links on webpages to build a synthetic RSS feed, both of which use LLMs, neither of which need a SOTA model.
I'll probably use LLMs to bootstrap a dataset of native ads in articles, and there again, I don't really need a SOTA model.
If it's for more open ended tasks like writing code though, I agree that at this point SOTA models make more sense to use.
I wonder where the market sizes will shake out for these different types of use cases? I am guessing right now 1 is bigger than 2 but not for long (by token volume)?
Experience, mostly, and bit of trial-and-error. For code I use SOTA, but for e.g. translations I use Gemini 3.5 flash, for some other use-cases I use Gemma 4.