AI Dungeon has already scaled up to over 1,000,000 players and managed costs. They have a whole article about it here:

https://medium.com/@aidungeon/how-we-scaled-ai-dungeon-2-to-...

An important note, as others have said, is that not every player gets the same model. Free players, as far as I'm aware, only get GPT-2 ("only" is a strong word, as up until GPT-3, GPT-2 was the state of the art in text generation and is still extremely impressive in its own right).