Comment by Rohansi | Hacker News Reader

Comment by Rohansi | Hacker News Reader

While I think the tech is cool, and I'm sure it is fast, the fact that the benchmarks make assumptions about your stack doesn't sit right with me. They even admit that it is the main contributor to its speed:

> The most significant (but not sole) reason SpacetimeDB is faster than other backends is that we have decreased the round trip time between your server and your database by at least 99.95%. In SpacetimeDB your application server, ORM, and database are merged into a single system, so all three of these are run within the same process. [1]

For most systems that is a fair assumption to make but SQLite blurs the lines by running in-process. They admitted that this makes a significant difference on X [2] but do not list those results on their benchmarks. It would ruin their "is that the X axis or is that the competition" line to include. I'm hung up on this because they market it for use in video game servers, with an emphasis on MMORPGs (they're making one). Literally nobody is using web APIs for networking in real-time multiplayer games. The benchmarks are completely meaningless for that use case.

On the topic of scale they have been testing sharding/IDC for a while in BitCraft Online (their MMORPG) but they peaked at less than 5,000 concurrent players [3]. That's small enough to comfortably run on a single machine. Especially if you know that their gameplay is 90% (or more) just waiting on timers to finish. The timers only progress when you're connected to the server so that CCU number includes all the players waiting on timers.

[1] https://spacetimedb.com/blog/benchmarking [2] https://x.com/spacetimedb/status/2027766319462904310 [3] https://steamcharts.com/app/3454650

I am the author of OPs article (SpacetimeDB cofounder). These are all fair caveats/criticisms.

> The benchmarks are completely meaningless for that use case.

I wouldn't say completely meaningless, but it isn't a game benchmark that's true. We feel very comfortable that it's the most performant backend for persistent games though. We were trying to show it's also more performant for web use cases.

> 90% (or more) just waiting on timers to finish

This is not correct. It's mostly processing player movement transactions. We do about 50 million an hour.

> We feel very comfortable that it's the most performant backend for persistent games though.

Depends how persistent you really need it to be. If rolling back a few minutes is tolerable in exceptional scenarios then I disagree. It will be more performant to keep state in engine and snapshot whatever you need to save every few minutes. That's pretty much what most games do now.

> This is not correct. It's mostly processing player movement transactions. We do about 50 million an hour.

I said gameplay, not processing time.

Yes, the irony is that vertically scaling will always go "faster" due to latency until you genuinely need more DB than a single box can handle. Premature horizontal scaling only increase complexity and reduce performance. And even when you genuinely need it the implementation has to fight merely to approach single box latency, which sounds like what these guys have done.

But also vertical scaling seems to be able to handle so much these days that more people probably aught to ask themselves if their application is ever likely to truly need horizontal scaling - Maybe my work is just a small pond, but I suspect we will eventually end up in a place where the vast majority of applications are happy on a single box, and only twitter scale things need to care about this stuff.

.. or maybe I lack imagination of what possible future applications might make use of such massive horizontal scaling on such capable individual boxes (seriously).

> Premature horizontal scaling

I feel like this isn't a thing when it comes to applications. All the decisions you'd make to allow an app to scale sideways are just, like, good design decisions? Even if you never went 1->2 instances. Otherwise every app would just be PostgREST on top of a beefy VM.

> I suspect we will eventually end up in a place where the vast majority of applications are happy on a single box, and only twitter scale things need to care about this stuff.

We're already there IMO. A single box can scale up so much more than people expect these days - over a thousand cores and terabytes of RAM. The people coming up with overcomplicated architectures and mandatory high availability get in the way of it.

A single box may have that much resources available, but no software will use them efficiently alone. You end up having issues with NUMA, with your pagetable growing significantly forcing you to deal with hugepages and eventually memory fragmentation, and who knows what other kind of lovecraftian horror coming from the sheer complexity of a single system with that much resources. Turns out scaling vertically also has cost. That's why eventually you still want to scale horizontally. And that's not overcomplicating things for the fun of it, it's just tradeoffs.

For sure, but scaling vertically to that extreme is also unnecessary for the vast majority of systems. You can still have over a hundred cores without needing NUMA. And most systems will never even need that much.

Reply on news.ycombinator.com