Details on EVE Online's server infrastructure | Hacker News Reader

Details on EVE Online's server infrastructure

eveonline.com

53 pointszumda14 comments

EVE has a single instance world which is a much more difficult problem than what WOW has to deal with. WOW basically caps the total amount of people that can be in one instance (or at least they used to, haven't played in a few years).

It kinda boils down to a massive supercomputer simulation where your limiting factor is the speed, but more specifically latency of communication between servers. Notice how this upgrade brings them far greater internal networking capabilities which should mitigate performance pressure from growth for the time being.

If you look at their hardware you can kinda glimpse how their software stack might function:

TQ Tech Details: (Not the whole system, just what runs TQ)

Servers 64 x IBM HS21 2x Dual Core 3.33GHz CPU's 32GB of RAM Each 1x72GB HDD Each

                2 x IBM X3850 M2's 
                2x Six Core 2.66GHz  
                128GB of RAM
                4 x 146GB HDD

Cores - 280 total Cores - ~1 THz

RAM - 2.3TB of Total RAM

Storage - 4.8TB of Local Storage - 2TB of SSD SAN - 256GB of RAM SAN

Network - Gigabit Ethernet - 4Gb/s Fiber Channel

EVE is also one of the major contributors to Stackless Python. The EVE client and apparently quite a lot of the server-side software are written in Stackless.

The WOW dungeons are instanced but the worlds aren't - in WOTLK there used to be a major problem with the new World Battleground (Wintergrasp) where hundreds of players would be playing in it at once and the entire zone would become unusable. Eventually they fixed that by instancing it.

WoW uses Shards to split the game world into several servers and then Instances to split off players from each Shard. Eve does the same thing with missions but only has one Shard.

The difference is if you meet a Eve player in real life your characters can meet in game, but in WoW it's unlikely for two random people to play on the same shard/server.

The game design makes that much easier to do. WoW has 4 continents with no loading screen (except for moving into instanced dungeons or battlegrounds) on each continent. By contrast, EVE is split into several thousand star systems. So while WoW has to put each continent on its own server, EVE can be as granular as to put each star system on its own server (though they usually use virtualization to run dozens of systems on one machine).

That's why WoW has to be sharded - each shard can only be split into 4 servers, whereas EVE naturally splits up by solar system.

It might look that way to a player, but if everyone on a continent moved to the same zone the game does crash. Also, you can't drag a mob vary far from its home area.

On the client side loading screens had more to do with loading textures for each zone to your graphics card than they did with handing off your character from one server to the next. You might not see a loading screen but if you gate home they still show you as the textures are loaded.

Gotta hand it to MMOs when it comes to architecture - they have some very unique problems to deal with. Sending media-heavy data back to a number of widely dispersed clients in real time has to take the cake in my book, not to mention continuously writing that same data back onto a database (you log in the same place as you logged out, don't you?)

I'd imagine the media messages they send to clients are a continuous broadcast of 3D coordinates and audio processing signals for player characters but I honestly have no idea. I'd love to learn more about it.

They don't send "media-heavy data" back to clients, actually. All the media comes with your installation DVD. Used to play Eve over dial-up.

/facepalm

The network has to send you coordinates about where you and every other player are in relation to each other so your view of the gameworld is synchronized. Had you actually read what I wrote you would have understood my meaning.

Before I had my daughter I played this game all the time. The politics, drama, and complexity of the game are very compelling. Their infrastructure and design always impressed me. Keeping the whole world in one instance makes for a lot more fun when playing with people you know.

Also a former player. They do have instances, but they are based on locations, like star systems. I think someone should rewrite the drive mechanics so that instances are attached to reference frames. (Which have a fictional physics upper limit on the number of ships that can join to prevent lag.) This way, a battle could rage in a particular system, but it would consist of many encounters. Such a system would load-balance itself!

Hehe, one of the most compelling features of Eve Online is inventing ways to make the game better. I'm sure I've spent more time discussing that on forums and with friends than I ever did playing it.

Then again, though the battles are extremely short it's the most exhilarating battle I've ever had in an online game, the death penalty is pretty harsh so you do get real adrenaline rushes. Pity they're always so hard to find. I can still remember most of my non-trivial ship losses.

Then again, though the battles are extremely short it's the most exhilarating battle I've ever had in an online game

Carriers/Dreadnoughts vs. a fleet of non-capital ships. This can go on for a very long time. If you're on the non-capital side, you can die quickly, but you'll still be listening in to an exciting event.

I get so used to my tens of thousands of cores that I forget the amazing things that can be done with far less . . . only two hundred cores! Only two terabytes of RAM! Astounding.

Reply on news.ycombinator.com