Show HN: Unity MMORPG Boilerplate – Multiplayer in Unity Made Easy
github.com
github.com
It turns out to build a solid solution that is largely overwhelming to individuals, and I yet to found an open-source framework that tackles these issues in a straightforward way.
To my shallow understanding, it's less about Unity, the game client, despite there are many protocol related works. Here are something I found really hard for me:
1. The first step is basically rebuilding all the game logic in the server-side, or you'll be facing endless cheating. The combat, the inventory, especially the map - you're going to model a huge 3D world on the server-side without existing client-side GUI helpers.
2. The most difficult thing is most of the states need to spread across the whole cluster like a distributed database, and keep sync because there might be interactions between those states (eg. shoot a fireball through the edge of the map to another map then hit someone), this usually done by modeling a quadtree across the cluster.
3. Players can randomly fight each other in an open world, and those combat needs to be dynamically grouped and disbanded. Players should only receive the state update they care about, instead of receiving every information of the world(area of interest), and everyone's interest might be different.
4. Of course, there are global clock and synchronization.
5. Many operations would require transactions, or players can trade and duplicate items when they cross the border of maps or directly disconnect.
Start with the server side. This doesn't actually have to be a server application with no gui, but start with the simulation of an entire world. Then 'rebuild' the logic on the client side. This lets you be a little bit sloppy—though not too sloppy, obviously—because it's ok if the server 'cheats', since it has no ulterior motives (except for yours) and there's only one of it.
Much easier to sync keys and play forward vs syncing at every Rand call.
Especially on an mmo, there is no point telling a player that his next fireball spell will also trigger a "random chance to put a burn status on target". The client would start casting a fireball, which takes a second to cast and in that 1 second duration the server would decide what is the damage and if the spell is going to apply burn and by end of 1 second, you would get that information so you can see the damage values and burn icon on enemy
It is better to keep clients dumb if possible
In the era it was used, server chips weren't really a thing. So it was really just "the current decent CPU."
It's also a fascinating historical artifact of "when internet play meant orders of magnitude higher latency and jitter... as a base assumption."
Per memory, there's a bit where they've ferreted out all the sources of randomness, except one. And any source of randomness corrupted the downstream calculations, as they used global random generators.
There was some reason they couldn't just trace calls directly.
It turned out there was a line of code somewhere that said something like "if {rare condition} then rand.next {to figure out where to place an ornamental rock/tree}."
So everyone has to have the same results and calculations at all times, because there's no authoritative simulation.
It's a common architecture for RTS's (interactions are far less numerous than individual entities) but not for general multiplayer games due to issues like everyone must be on the same timestep at all times -- so a single slow player slows everyone down. See https://gafferongames.com/post/deterministic_lockstep/
If you have an authoritative server, so the client is mostly a presentation/view of current game state, you don't need to be as careful about maintaining game state/randomness, since there's only one source of randomness (your authoritative server)
Ofc, if in an mmo you have multiple authoritative servers, you're back to the same problem, though I believe the more common solution is to have 1 server - 1 universe, and you just have more game instances (shards/servers) to connect to
E.g. grid & turn-based vs 3-D free-motion worlds
They say they specifically made the deterministic lockstep choice because it was the only option that (a) allowed the multiplayer gameplay they wanted and (b) fit within the computing and network resources of the time.
once a combat session is initiated you don't need that much prediction going on because where you move is uncorrelated to where a fireball hit or not, only player stats.
the real exception are are effect, as the client need to reconcile them, but the client can chat as much as it wants as long as the presentation matches the server player stats: it can for example position area effect that hit on the server in a position such as the player hit was in range.
the client only had to make sure action result are in sync, not the world physical station whole
Usually these mechanics are not as advanced and detailed as in single-player action games, but they do exist.
To improve the experience and compensate for lag player clients run a simulation and can update their state based on their player actions directly before the action is registered or acknowledged by the server client then they only get a confirmation from the server or they get a forced update if the action they performed wasn’t in agreement with the server.
As for other things like give mentioned you don’t usually have to run a full model on the server side of your world, you however hold a representation of it in the form of maps, these are usually huge arrays of coordinates with various properties they basically transform the 3D game world into a board game board.
For MMO’s specifically while there aren’t many good open source frameworks there is something quite good and that is private server clients for popular MMO’s like Mangos https://github.com/mangoszero/server/blob/master/README.md
I think it's because of a combination of technical difficulty and game design reasons dealing with overcrowding and camping and other bad behavior.
And I also think it's a total cop out and decimates any sense of "place" in these games, which I think is so important in games like these.
If anyone has examples of modern games that don't do most of these these things, I'd love to hear it. I'm not up on all of em, particularly the ones in the primarily Asian market.
It seems to be impossible to handle unlimited players that are crowded together, even despite all the design issues. Because the logic and synchronization could grow in superlinear complexities in those scenarios. Clustered states and dynamic instance provisioning could ease some degrees but that actually can only solve sublinear to linear complexity problems at most.
With that being said, I played the Elder Scrolls Online many years ago. There were definitely phasing mechanisms. But when it just came out, there were like one thousand users PVP in Cyrodiil. Compared to World of Warcraft which one hundred players AOE in a city could make the server crawling it's already pretty impressive to me. Not sure how they have done it.
Within a system, your area of interest is always the grid you are in, which is equal for all players in that grid. "Grid" despite the name is a dynamic boundary that contains all nearby players that are likely to interact with each other. The game doesn't allow most cross-grid interactions, probably to make implementation much easier. I think they get away with this since solar systems are so large and you have to be relatively close to interact with anything.
Some of these designs became problematic.
Players can perform "grid-hacking" where they manipulate the shape and position of these grids so that they can suddenly cross a grid boundary to ambush or escape other players.
I haven't played in a while, so some of this might work differently now. The lack of server scalability within a system also causes time dilation, which besides making fights slower and boring also gives more people enough time to join and rejoin fights and cause more time dilation.
The problem was when everyone decides to hang out in one shard.
* You checked in your .vs folder, my understanding is that it is the .suo equivalent and shouldn't be checked in.
* Pulling updates from the GitHub releases page will probably get your repository rate-limited if not taken down, you need to think carefully about how to scale that up.
* While the readme says it pulls updates from GitHub, I can't find any code that actually does it. As far as I can tell the Launcher folder is completely empty boilerplate.
* Using XAML for the launcher UI seems like it would negatively impact portability unless you're pairing it with something that supports Mac and Linux (I couldn't tell if you are). It would make a lot more sense to implement your launcher in Unity (as a separate executable) since the end-user already is using Unity.
* You appear to have checked in local copies of NuGet packages, which is a bad idea. You should gitignore that folder so developers' msbuild can restore the packages locally.
* It looks like you also checked in third-party Unity asset files? I'm not sure you should do that either.
* You appear to not be making use of indexes for many of your queries, for example:
var user = db.Users.ToList().Find(x => x.Name.Equals(name));
This will stop scaling almost instantly.
It's even worse. This will literally retrieve all Users into memory and then iterate over it. Every time you want to find this single one. Looks like the person who wrote it never actually wrote a single backend server using Entity Framework. It also does so synchronously, while async method is available. This means that this thread will be locked until it finishes this IO and won't be able to process anything else.
I think FFXIV has a hard cap of around ~350 per zone, but you get some silly processing outcomes when everyone is in the same area (entities or attacks not being rendered, area of effect attacks randomly only hitting a portion of people within the target zone, etc).
You are right, definition of MMO is stretched quite a bit when you have such limitations. One could argue that counter strike is an MMO because you thousands of players are playing on same universe (I guess gun skins are the only real progress of the game :) ) and each server is an instance. I think calling something an MMO mostly depends on how well it is hiding that it is not really a MMO
350 sounds quite impressive. Looks like eve online handled 2,670 at some point: https://en.wikipedia.org/wiki/Bloodbath_of_B-R5RB . eve online slowdowns the server to a crawl when there are such large scale battles though
The funny thing is that the maximum number of people on the screen before performance issues really hasn't improved much in MMOs in 16+ years.
They're probably pumping up the graphical effects and maintaining a stasis there. But there are also hard caps to how much data or how many position updates you want to be sending in packets before perceived latency degrades too.
FFXI in particular had a famous meme "PS2 limitations" where the game was being held back technically due to supporting the older platform, not to mention it was built with dial up internet in mind and a very low update rate. Many MMOs in general want to cast a wide net and will rarely improve to the point where players with older systems can't join in. That's for the sequel!
In practice it wasn't utterly horrible to play this way, it was just very surreal to be running around engaging in combat with name-plates floating in the void.
But, that method is difficult to get right in Unreal Engine and impossible to do with Unity. That's why you only see it in high end titles like Battlefield but not in any indie MMORPG.
This is the default mechanism for electron apps to auto update, and I haven't heard any issues from it to do with rate limiting.
The launcher uses (Avalonia) which is cross-platform (Windows-Linux-MacOS), not sure about mobile though. Now that I think about it, I suppose there could be different ways to go about mobile and just avoid the launcher entirely.
Works pretty well, even in massive fights with hundreds of people involved. Frankly handles the MMO part of things better than most MMOs.
ENet provides a middle line between reliable and unreliable packets making it perfect for MMORPGs. You want to send a packet that a player shot a bullet? Use reliable. You want to send position updates? Use unreliable.
I'm guessing you can use timestamp by having PlayerA and PlayerB send udp packets, but wouldn't that cause latency, even if you put the instance region zones close enough to both players? How about direct socket connection between PlayerA and PlayerB and sending some results to the backend, but how do you prevent cheating on the client side then?
Because cheating tends to be an outlier activity, some companies choose to invest more in identifying cheaters after the fact using pattern recognition and machine learning tools rather than outright prevention. Depends on if your game inherently resets between game loops (i.e. battle royale, deathmatch, MOBA, etc), or if you have a continuous persistent state that has to be kept stable (i.e. MMORPG).
Full security means validating every activity with a fully trusted server before broadcasting to other players. Full performance basically means full client trust, whatever the implementation ends up looking like.
There's a whole spectrum of solutions somewhere in the middle for most games.
If you can solve the latter problem in a general way, you can easily build a company around it. There's already a bunch that do nothing but this. It's just a fundamentally hard problem because the speed of light is what it is, the distribution of computing hardware is what it is, and the distribution of players is what it is.
Some cheaters try to fool server/other clients. For example in a game of chess, a client can try to move his pawn like a queen. In this case a server would run the "game simulation" it self and would detect that the client is making an illegal move and a cheater. But this is not always possible/feasible. In an mmo for example it may not be feasible to run physics simulations on server and maybe a server can trust clients when they say "I moved from my old point A to new point B" but in reality there may be a wall in between point A and B
Not all cheats are about fooling the server. Some runs on entirely on cheater's side. A board game example would be battleships. A cheater can decide to "look" other side of the board and cheat by looking at opponent's ships. This kind of cheats are solved by only sharing necessary information with clients. For battleships, this would be server only responding the bombing coordinates a client sends and the answer would be a simple "hit/miss". Again, this is not always practical. In a game like counter strike, ideally you would share enemy positions only when they should be visible on someone's screen. But due to lag or not being able to detect this visibility information, a server can choose to always share this information and clients can cheat by presenting that information
A harder to catch cheat would be aim hack. In this case a cheater neither fooling the server or benefiting of an information it doesn't have. They usually catch these by checking programs that does this or via player reports or automated systems that catches players that are too good to be legit.
Edit: As cheschire said, it entirely depends on the type of game and what kind of actions the player is performing
Rule of thumb that is used in banking, 'sanity check'. Cheap and effective way to see if the gains-number (e.g. xp/coin gain) is making sense. If a top player can make 100k per hour and you get a newbie making 500k/h, you better start reading logs/snooping. The value of equip when a player saves & exits would not be x100000 from the previous time he/she saved and exited (and other similar metrics).
It is an ongoing process to make and maintain a useful dashboard. Also (I've seen it happening in some MUDs), you start snooping random players at times (preferably ones with massive gains) and see what is what.
Imho the cheapest way is to detect and correct, not prevent cheating. Unless you have an army of devs scouring the internet looking for cheating solutions that can work out preventative methods.
One way of doing that is that the server runs the authoritative simulation and the clients send input to the server, but the clients also run their own simulations. In some games the client will try to predict what the server will do. If there's a conflict with what the client simulates compared to the server then the client gets rolled back to the server's state. This usually happens when there's a second player that interferes in some way.
Here is a talk by Timothy Ford about Overwatch's netcode (it's really fascinating): https://www.youtube.com/watch?v=W3aieHjyNvw
https://arstechnica.com/gaming/2019/10/explaining-how-fighti...
While it's mostly about ensuring both clients have a smooth experience and how to achieve that (prediction, rollback, etc) many of the same techniques apply to cheat prevention - have the client "echo" or predict what it thinks will happen, but have the source of truth be the server. "Desync" is a common term for what happens when they disagree, the server keeps its version of events and kills the client connection. In peer-to-peer or direct connection games, one player is the server and this does present a problem, but keeping it random will reveal statistical irregularities.
In general, how it works depends on the cheat. You cannot stop someone running a program to automatically aim and shoot at enemies. The enemy data is in memory so pinpoint accuracy is simple to achieve. Usually, statistical analysis, recording, allowing an admin to view their screen, and other methods can be used. Often, anti-cheat software is installed to scan for known cheats or by using heuristics (much like anti-virus software). This is a bit of a hot topic right now as a recent game tried to install such a program in Ring 0 (lowest level device driver) in Windows, which runs at startup even if the game isn't running, and won't automatically uninstall. Some people reported it stopped their PC booting.
For informational cheats (such as seeing players through walls) that data is also in memory and is harder to detect if the player doesn't make any illogical plays taking advantage of it. You might ask why the client has player position data for people you can't see? That again depends, but IIRC in at least one case a game tried to suppress the information until just before they became visible but the cheaters realised that the game had to generate sound (footsteps) for the players and reverse engineered the directional sound back into positions.
In other cases, passing network data through a proxy defeats local anti-cheat. Packet scanning and direct injection can be used for superhuman speed (such as claiming unique monsters in an MMO before anyone else can).
In general, stopping cheats is largely impossible, it's more about how to catch them and policies to handle the fallout e.g. If I cheat and earn a bunch of ingame money then trade it to you, should you lose it? How do we know you're not an accomplice?
There is also a MMO called Pwn Adventure where cheating is intended for you to progress. Live Overflow did a series on it at https://www.youtube.com/watch?v=RDZnlcnmPUA
I'd been "planning on building a game" for probably a decade. Pico-8 got me to finally "finish" a few. The integrated graphics and audio system is super useful for cobbling something together.
More important than any particular route you take is merely sticking with it. The choice of game engine doesn't matter that much as long as you make and finish games with that engine. Everything else is secondary and mostly procrastination.
After there is the big engines, Unity, Unreal, and Godot. They all have their ups and downs, but are the next level of complexity up from the previously mentioned tools.
If you're into specific niches there are sometimes purpose built engines for it, like RPG Maker. They are usually much less complex, albeit limited to specific kinds of games.
This is a popular enough way of doing it that it exists for most of the popular engines. Unity just tends to have the most.
You can start here: https://www.youtube.com/watch?v=j48LtUkZRjU&list=PLPV2KyIb3j...
Using a game engine like Unity you can create something pretty advances in a short time. There are tons of tutorials on YT. If you prefer a free game engine, check out Godot.
If you're looking at doing it "from scratch", you should really consider starting out small with something like Pong or Tetris. The best resource I know is Handmade Hero for that, tons of YT videos and huge community.
Why almost all links are about years ago? Are not there any new technologies/methodologies for MMOGs that support 90fps, or even more? Something hybrid which supports both server and client sides? And mostly has developed for Unreal Engine or other well-known game engines?
I do authentication from another service and send encrypted commands to the server which carries them out and encrypts the responses back.
Even for a text based game it is tough but I keep it simple as long as I can!