with some basic extrapolation, at how many users they'll hit the extra expensive Hertzner servers? or in other words, at which point they'll need to improve their architecture?
Can we please drop the "number must go up" mentality? The whole point of federated systems is to avoid concentration of power in a handful of servers. I'm sure that the people doing there have good intentions, but why can't we just let things just a little bit dispersed?
Besides, that's not the point. The point is that you don't need to have a Gmail account to reach and be reached by Gmail users.
Thankfully, there are proposals to add ActivityPub as a web API: https://github.com/webap-api/webap-browser-extension
Its "community" (on Lemmy), or "magazine" (on kbin) scaling that seems hard.
But since each server has a local-copy of the community that its serving out, maybe the hardest part has already been solved by the Federation model. Each federated-instance is effectively a proxy / front-end for the users on that instance.
---------
I guess Mastodon is way larger than Lemmy though and they haven't had issues yet.
2TB RAM is common in commodity servers, albeit expensive ones ($1000ish/month). Somewhere between 4TB to 20TB RAM is the pragmatic limit (where costs for vertical scaling start to get far worse)
No one knows for sure until we reach those caps.