Hacker News Reader

Comment by dbatten | Hacker News Reader

Reminds me of the excellent post about the architecture of multi-player in the original Age of Empires game:

https://www.gamasutra.com/view/feature/131503/1500_archers_o...

You had high-ping 28.8k internet, and you wanted to be able to have immersive battles in a world with hundreds of units controlled by several players. There's no way you could send real-time information about the status of every object in the game. So they didn't. They made the game simulation completely deterministic and then simply shipped user input around on the network. User input was processed on a slight (couple hundred milliseconds) delay (to allow for network transmission speeds) and executed simultaneously in "game time" on everybody's machine, so the simulation stayed in perfect sync. Voila, no worrying about lag.

View on HN

Fighting games take this one step further through "rollback" netcode. Instead of delaying the game while waiting, the game predicts what the opponent's input is going to be. If it turns out when the input arrives that the prediction was wrong, the game state is rolled back, the correct input is applied, and the game fast-forwarded to where it was.

Here's an extensive writeup about the concept:

https://ki.infil.net/w02-netcode.html

Many realtime online games use the same techniques today. For a very understandable walkthrough of the concept, I recommend Gabriel Gambretta's excellent 4 part series: https://gabrielgambetta.com/client-server-game-architecture....

Interestingly enough, your brain does the similar things, playing games with temporal relationships and fixing it all back up after the fact in a way that you normally can't notice.

https://en.wikipedia.org/wiki/Chronostasis

I was talking to someone about a related topic a couple weeks ago, how the brain "fudges" with the experience of time to compensate for the latency of perception. But I couldn't remember what it was called.

Thanks to the link above, I found the exact phrase: neural antedating/backdating. Ah, it's nice to have resolved that question.

Except Super Smash Bros, which has the worst netcode and online play possible. Nintendo is this weird combination of incompetent and brilliant that I don't really understand.

Nintendo's track record with online is truly bizarre.

They clearly have some phenomenal tech chops in the company. Iwata hand rolled a compression algorithm and Nintendo were in the internet game waaaaay early and had aspirations of being an ISP with their Famicom Modem in the 80s.

That said their web infra just seems so janky and backwards - I've heard excuses like the friction wrt online gaming keeps it family friendly and helps disincentive trolling.

Nintendo has also always been more focused on in person social which works well in Japan where console market penetration is very high, gaming in public is normal (both socially and public crime/mugging doesn't exist) and population density is high. Monhun (Monster Hunter culture) being a great example of JP cultural gaming trends that didn't export nearly so well outside of Japan. (did ok-ish in Asia). [--aside: Must have been so ace to have monhun parties after work /sigh--]

If Nintendo could make their estore more optimized for UX (think Amazon or Apple App Store / Google Play Store) I feel like they would be a money printing machine.

Of Sony, MS, Nintendo, Nintendo are the only pure games company and those marketplaces are fantastic cash cows...

I remember hearing/reading when Slippi (community rollback netcode implementation for Melee) came out the creator mentioning that Japanese fighting games don't bother with rollback because it's more work and Japan is small enough and well wired enough that it's never a problem. Contrast with fighting games that come out of the US that normally implement rollback.

Yeah the Japanese seem to operate under the impression that the entire world lives in Japan. I'm a semi-pro Super Smash player and I've desperately wanted to play it online, since like forever. But every version, the lag is unbelievable. It's always some p2p bullshit where one person lags, everyone lags.

I remember when the Wii U Smash was coming under criticism for the unbearable lag, the developers released a statement in which they literally said: we tested the online play in development, and it works great. Are you fucking serious? How can they care so little?

It should be noted that Japan is not the only country that frequently disregards technical constraints, units of measurement and other cultural circumstances from outside one's own country or seaside metropolitan area.

I recall that Supreme Commander used the same technique, a fully deterministic sim run in lockstep by all multiplayer peers. For it's era, it had very very high scale for an RTS game. (thousands of units and enormous game maps (the biggest of which took 15+ mins to send naval units across)

If I recall correctly, there was another benefit. You could save a multiplayer game by just having all peers dump the current game state to disk and resume it later. Since a big SupCom game could take HOURS this was a great feature.

However, there were 2 issues with SupCom's implementation.

1. It wasn't perfect and sometime hours deep into a big game, you would get a "desync" error and it was unrecoverable.

2. Everyone needed to be able to run the sim at the same speed. So the game could only progress at the speed of the slowest player's computer. And SupCom was VERY compute intensive.

I had no idea that supcom worked that way! Thanks for the insight.

Your comment brings back fond memories of playing huge supcom games at a local LAN cafe. Those were some of the best times I've had gaming.

I couldn't count the number of hours I spent playing that game with friends. Never played anything else like it!

> For it's era, it had very very high scale for an RTS game

> So the game could only progress at the speed of the slowest player's computer. And SupCom was VERY compute intensive.

It might surprise you to learn that these statements are still true! SupCom:FA (still playable by buying a copy and through the third party FAF launcher; they have even fixed most of the desyncs, it's very stable these days) is probably still the largest scale RTS out there (even planetary annihilation only feels larger). And it still requires a top-tier gaming rig to play.

The reason for this is pretty straightforward, CPUs actually haven't gotten faster for well programmed, cache aware, tight-looped CPU bound operations in the last 15 years. And in fact there are arguments for why something like SupCom:FA would be difficult to make today given the industry's reliance on game engines. And games are still rarely threaded more than SupCom was (e.g. game logic thread, render thread), so until someone can get back to the low level excellence of SupCom:FA and then design and implement a concurrent game logic loop for it that actually parallelizes well, I think it will remain the king of high scale RTS for quite a while.

https://youtu.be/DEhp5eCwgSI?t=3304 (to see why this game is awesome)

Total Annihilation (unsurprisingly) had the same speed-limited-by-slowest-computer thing so I would imagine it was similar.

It always made me a bit sad that the comparatively low-scale Starcraft seemed to suck up so much of the space in the genre.

StarCraft 2, the modern "king of RTS games" so to speak, still uses that model! Determinism based on user input, plus a deliberate lag (~150ms?) in handling user input to allow time for player inputs to be received.

And the original Starcraft 1 was the pioneer of the technique fwiw. They even let you set the input delay in the options to account for lag.

Aren't we saying that all of the popular 'Real Time Strategy" games are in fact turn-based, but there are 7 turns per second?

Hah, nope. A delay of 150ms doesn't mean that events only happen every 150ms. Also even if events were sent in 150ms chunks, the offset of time within that chunk would still matter.

With that said, most games do have a maximum tick rate, so in a very misleading way you could say they are turn-based ;)

If we keep a complete record of all player inputs we can also replay the exact game after the fact, a great feature that arises as a consequence of this design choice.

A great feature not only for players, but for developers as well. If every change to game state goes through an event system you can reproduce any bug or crash as long as you're careful about making sure that the event stream is recorded before the process is killed.

This ended up being an enabling technology for e-sports too, didn't it?

The observer gets the replay log (cheap), and broadcasts the pixels out (expensive) to the world with very little impact on the game play for the active participants.

No one watches sc2 matches by loading up the game and running a live replay -- it's all video streaming/streamcasting. Casters do use a live (delayed) replay, letting them poke at any arbitrary object in the game, but a video stream of that is what gets broadcasted.

Low bandwidth broadcast isn't really relevant I think to esport success -- the "complete" replay aspect to even 1 viewer (the caster) however might be

I think you're in agreement with the post you're replying to.

You both say that the game state is sent to a non-player machine that creates the actual video stream seen by the viewers.

Sort of -- im saying that for streaming, this doesnt much; deterministic lockstop does little for the general viewership, and saves relatively little processing compared to recording the stream directly on the player's machines.

Much more notable is the impact it has on casting, where it's presence is clearly felt -- casters are granted much more freedom, and can be much more thorough and descriptive, when complete replays are available (versus simple dumb video).

We agree about the technogoly -- we disagree on its impact

I built an implementation of this architecture in Python if you want to see how it works in code:

https://github.com/eamonnmr/OpenLockStep

Somewhat related, one of the developers discussed some of the issues he faced when working with the re-mastered "Age of Empires: Definitive Edition". https://richg42.blogspot.com/2018/02/some-lessons-learned-wh...

That is what you try to do with most games even today, just to make it harder to cheat, and various other reasons (easy to replay games, its the natural thing to do, etc)

There are times gameplay needs don't allow that model to be perfect, but it is the norm.

Afaik its only the norm in RTS games -- almost everything else is client/server, with the server providing (and authority of) full game state. In which case the client is just a view into server simulation -- logic moved to the client only so far as to enable faster input response through local interpolation (with provisions to undo, if the server decides thats actually wasn't allowed), and that's also where information may be leaked to enable cheating.

The main reason for the deterministic lockstep architecture is the cost of sending game state -- in a game with 10 player characters and some particles, it's sufficiently cheap. In a game with 10,000 player characters, not so much.

Determinism is a bitch to maintain, so afaik, no one tries to do so unless they must.

Nice! I used this approach for a turn based gamed engine, but with the input processing lag it totally makes sense for real-time games as well!

Reply on news.ycombinator.com