"How dumb do you want your AI to be?"
"Why do I have to select?"
"Because smart costs money"
"So... I can have dumb AI but it's cheaper?"
"Yes"
"How would the average person know which to pick?"
"Oh you can't know."
I hope they can invent an AI that knows which AI model my question should target cheaply.
ChatGPT lets me choose between GPT-4o ("Great for most tasks"), o3 ("Uses advanced reasoning"), o4-mini ("Fastest at advanced reasoning"), and o4-mini-high ("Great at coding and visual reasoning").
Is what I'm doing "most tasks"? How do I know when I want "advanced reasoning"? Great, I want advanced reasoning, so I should choose the faster one with the higher version number, right? etc.
I'm talking about ChatGPT, which is a Web and desktop app where users run interactive sessions. What does "production" mean in this sense?
- there are billions of people who are not accustomed to using software this way, who are in the expected target market for this software. Most people cannot tell you the major version number of their mobile OS.
- this approach requires each individual to routinely perform experiments with the expanding firmament of models and versions. This is obviously user-hostile.
Anyway, my hot take here is that making things easier for users is better. I understand that is controversial on this site.
Anthropic, by contrast, uses the same name for the user-facing app and the models. This is confusing, because the user-facing apps have capabilities not native to the models themselves.
Some users may not appreciate that, but many more might be drawn to the "adjust the color balance on the TV" vibes.
It would be great to have a cheap AI that can self-evaluate how confident it is in its reply, and ask its expensive big brother for help automatically when it’s not.
Shouldn’t asking a more expensive model for input be a similar level of «tool use»?
If paying by API use it probably makes more sense to talk to a lesser stooge where possible, but for a standard pro plan I just find the lesser models aren't worth the time to use in frustration they cause.
I don’t think I’d trust the vendor’s ai to optimize when they will likely bias toward revenue. So a good case for a local ai that only has my best interests at heart.
Currently, the guidance from vendors is “try it and see which yields the best results” which is kind of like “buy this book, read it, and see if you like it” and how of course the publisher wants you to take this action because they get their money.
Isn't that the idea of OpenRouter?
I think you are getting confused by the term "Model Routing", which to be fair OpenRouter does support, but it's a secondary feature and it's not their business focus. Actually OpenRouter is more focused on helping you choose the best provider for a specific open model based on their history of price, speed, reliability, privacy...
The model routing is simply provided by NotDiamond.ai, there are a number of other startups in this space.