I personally prefer per-token, it makes you more thoughtful about your setup and usage, instead of spray and pray.
You can also access the notable open weight models with VertexAI, only need to change the model id string.
I personally prefer per-token, it makes you more thoughtful about your setup and usage, instead of spray and pray.
You can also access the notable open weight models with VertexAI, only need to change the model id string.
However, from a game theory perspective, when there's a subscription, the model makers are incentivized to maximize problem solving in the minimum amount of tokens. With per-token pricing, the incentive is to maximize problem solving while increasing token usage.
I do agree that Big Ai has misaligned incentives with users, generally speaking. This is why I per-token with a custom agent stack.
I suspect the game theoretic aspects come into play more with the quantizing. I have not (anecdotally) experienced this in my API based, per-token usage. I.e. I'm getting what I pay for.
Any tips?