At the beginning of my career, I incurred some hundreds of dollars in losses with BigQuery and Google Cloud Function. The problem with these services is that they are easy and intuitive enough for a beginner to use, but a nightmare to maintain.
It was originally estimated like 10k or something per test which was approved at the time (had like 3 level of management all down my neck for getting it out, hence using lambda originally).
We did deliver, just needed one more sprint to rewrite it as a distributed system on servers. ;) Moved to like 20 machines w/ 128gb of ram that we could spin up as needed (testing millions of events a second, system in NodeJS!)
But really soon I noticed the slow startup … simple queries took too long (1.2 sec vs milliseconds in a traditional database)
Then I learned a lot about BigQuery views. That helped a little.
At some point I simply wanted to export data. New Google tools needed to be learned: Cloud Storage, Data Flow.
After 18 months of using BigQuery on roughly 850 million rows, I switched back to a traditional database.
My record is $20k and it raised some eyebrows. But it was not really a mistake, just a sub-optimal backfill.
The data was filling a need for making appropriate business decisions, and compared to all the money lost by business developers making investments on a hunch, this was a very small bump in the road.