Hacker News Reader

Comment by throwaway894345 | Hacker News Reader

As someone who writes a lot of server code, I've always been curious about the architecture for MMOs--how do you handle issues of scale (or is it fine to assume everything is running on the same server)? If you do concern yourself with scale, presumably you have to "shard" your universe somehow? Do you shard by in-game geographical regions? If so, how do you handle players moving from one region (shard) to another? What is the best practice for handling game state? A SQL database? A redis server? Or maybe you just keep things in memory and persist asynchronously? Can anyone recommend blog posts or similar on these subjects?

View on HN

Fun fact: the word "shard" in this context was coined by the dev team of the first major commercial MMO, Ultima Online, as an in-universe explanation for why the game world was split into multiple servers.

If you're looking for an exemplary server architecture for MMOs, look no further than UO. That game came out in 1997 and supported worlds with thousands of players and millions of persistent objects. Players could build their own houses within the game world, decorate (and later design) them to an incredible degree [1], and have other players visit (welcome or not). Players could run their own shops within their homes and sell their own hand-crafted, signed goods (weapons, armour, clothing, furniture) in addition to any found treasure they wanted to offer.

What made UO so impressive, technically, was how they accomplished all of this on such modest hardware as was available in 1995 [2]. This second link is to the GDC postmortem of the game. If you're interested, I recommend you check it out. It may not answer all your technical questions but it's extremely interesting nonetheless. The story of how they had to shard the game is part of the talk.

I'm pretty sure they did not use any off-the-shelf databases as the performance would have been terrible at the time. Everything would have been custom. The custom networking protocol is extremely reserved (not chatty), since it was designed for dial-up modems. Even when you run the game on modern hardware with broadband, the game uses on the order of a few megabytes per day of traffic.

[1] https://duckduckgo.com/?q=uo+house+decoration&t=osx&iax=imag...

[2] https://www.youtube.com/watch?v=lnnsDi7Sxq0

According to WikiPedia, it does say that Ultima really did influence the use of "sharding" as a term in databases: https://en.wikipedia.org/wiki/Shard_(database_architecture).

I have heard Richard Garriott claim credit for coining "shard," and I'm very skeptical. The first database paper I could find that uses "shard" (as an acronym: System for Highly Available Replicated DAta) is from 1986: https://dl.acm.org/doi/book/10.5555/889956

Those authors may have been first, but I seriously doubt they were a significant influence on the term's popularity. If there's one thing I've learned from reading a lot of CS papers, it's that pretty much any word you pick out of a hat has been used as an acronym by somebody.

The paper you link to doesn't use "shard" as a noun, and in fact it doesn't describe a system that has "shards" in the sense that UO and just about everyone else since then has understood that term, i.e. independent partitions of a dataset:

> The reader is referred to [SBK] for a detailed description of the architecture of the SHARD system. Briefly, the main ideas are as follows. The network consists of a collection of nodes, each of which has a copy of the complete database.

(emphasis mine)

Additionally, according to Google Scholar, that paper was only ever cited 21 times. I checked the 16 of those citations for which full-text is available, and not a single one of them used the word "shard" as anything other than a passing reference to the name of the system itself.

Welp, looks like I'll be going down this rabbit hole all weekend. Thanks!

Oh this is cool. Thanks.

For most Indie MMOs you can use a single server. You also store game state in memory and checkpoint it to disk like a DB, instead of having a DB as a bottleneck. This can scale to many hundreds of thousands of concurrent users if you do it right and it's easier than building a distributed system.

Scaling it beyond that (you're very lucky!) then involves sharding, and you can build a master/slave system that splits shards and moved them around as needed.

Source: working on my own MMO which is a realtime location based tower defense game.

https://blog.winricklabs.com/(02-17-2020)---efficient-data-s...

All those questions are answered on github in what probably is the smallest (as a feature, not a flaw) MMO system codebase in existence. Short answer is I made my own HTTP server from scratch with distributed database over HTTP and everything runs on top of that. Sharding on separate machines is not interesting because the whole point is to play together, no matter how; you always want to be connected somehow. Larger teams separate the services and it becomes really complex. I want to be able to develop and maintain a massive MMO on my own so I had to improvise.

That said, I made a subreddit for the topic: https://www.reddit.com/r/mmodev/ it's kinda empty now but hopefully people will contribute eventually.

I am under the impression that you write on the database using HTTP get/post methods. Am I correct? Is it feasible for real-time games?

Persistence often uses much slower methods, one nice feature of my database is that it is distributed async. so I send all database queries to all regions which means I get backup and players can play anywhere without transferring data first.

I also use HTTP for real-time movement, you can do that if you strip all non-mandatory headers and use "Transfer-Encoding: Chunked" for the "Pull" socket. I use two sockets per client: "Push" (Client <-> Server) and "Pull" (Client <- Server).

HTTP uses TCP and that will inevitably lead you to the old TCP vs. UDP threads. My take on it is that TCP works fine for real-time action games today if you use event based protocols instead of tick based!

Link to github?

https://github.com/tinspin/fuse

Reply on news.ycombinator.com