This needs a title change or deserves to be flagged. This is an estimate by someone on the outside doing a bunch, as he describes it, "back-of-the-envelope calculation.".

To post this with the current title of "Cost of running ChatGPT is $100K per day, or $3M per month" is absurd.

My brain had a tough time aligning those two numbers (the former felt very expensive and the latter felt small), possibly because I think about money in $XK amounts as "personal" money and money in $Xmm amounts as "business" (or large organizational) money.

The assumptions in the thread are ok, but is it possible that the 15-20 words/s that they mention is because they are batching inference?

I am not surprised, I've done thousands of prompts at this point and I know how expensive that hardware is.

To give myself some context, for $3,000,000 that is 1,600 A100 GPUs across 200 servers on GCP, 24/7.

...Based on assumptions about architecture, hosting, price, users, and queries per user.

Add "Estimated" to the title

I can't wait for the bucket brigade to come in and chastise OpenAI for letting compute resources go idle