Relevant post: https://modal.com/blog/dollars-per-token-considered-harmful
(disclaimer: I work with the author)
(disclaimer: I work with the author)
1. Estimate free invoicing the requests and letting users figure it out after the fact. 2. Somehow estimating cost and telling users how much a request will cost.
We have 1, we want 2.