Consider something like a "select * from bigtable" SQL query that might process a trillion rows. Hard to know that's going to cost $100 until after you have run it.
I recently had a debate with a colleague on this topic but concerning estimating the costs of AI agent work. For example, if you prompt an AI to refactor your codebase, the final cost can't be estimated perfectly, but I'm sure it can at least be estimated with some amount of precision! Like simply knowing that it will cost < $100 is actually great information even if the final work only ends up costing $5.
I think there actually might be a business opportunity (or at least the opportunity to build something cool here) if anyone wants to work in the AI cost estimation space. It's not exactly an idea I want to pursue, but just thought I'd put it out there. AI cost estimation (even with wide confidence bands) would be very useful to a lot of people.
Green means go Orange means finish what you're doing but don't start anything new Red means stop everything
And probably a special rule to permit stable, critical spend through regardless, the same way we allow police and ambulance to run lights.
Suddenly a database, a storage service or a computer service needs to be aware of the billing situations and make behavioral decisions based on the billing status. Again, not impossible, but something that suddenly promotes billing from an async/non-crucial background service that can be paused, replayed, adjusted by account teams etc, into a crucial hot-path service.
Not to mention having no way to network isolate such service as every single service in every single network boundary needs to be able to contact such service, and every service needs (more or less) same auth permissions on all user accounts it’s running.
Again, solvable problems with enough code, but not simple problems by any means if you operate at a large scale.
If you stop then you have to decide whether to charge for uncompleted work.
Interesting tradeoffs.
For very small ops e.g. individual Lambda invocation you have similar concerns especially if lots are fired at once from a queue or schedule or fanout.