Hacker News Reader

Comment by rafaelmn | Hacker News Reader

You don't have to spread your payment info to 10 different places ? You pay a fee for centralized API credits

Payment processing for tokens - sounds like the thing Stripe does.

Why do you need 10? One or two fallbacks is sufficent. Maybe my workloads are vastly different than yours but I’m not a huge fan of having to switch between models. Even within a single model determinism sucks, debugging why something worked yesterday and doesn’t work today because a model switch is not fun. I can’t imagine the pain of having our services flip randomly between 10 different models.

It is changing so often that to keep the costs and quality at a pareto level you need to experiment with a lot of different providers. And the pain you are describing should be a normal process in AI. I use model pools where the tasks are optimized for multiple AI providers. For example I was experimenting with GPT Luna and it turned out that the model is good but tool shy so I had to improve the instructions. Now this model is my main model for the chat in my app. Next week it can be a different model.

It might just not be for you. I use OpenRouter because I do like being able to quickly check whether a new model works better, but mine is a “human in the loop” dev process so it won’t ruin a day of batch processing or anything.

I do like being able to try eg GLM without having to set up a new account. It’s also nice that I don’t have to top up per-model accounts. I think I have a couple accounts with $7 in API credits sitting around.

Probably not something I would do for actual business processing, where the stability is dramatically more important than tinkering with new models.

Reply on news.ycombinator.com