Thundernetes makes it easy to run your game servers on Kubernetes

Is this effectively v3 of Thunderhead? Good to see the pivot towards using k8s, the 20+ minute propping of a 1-2GB container on vanilla Playfab/THv2 really made it a non starter for our company's internal playtesting.

Also is Hesky still around these days?

Hi. Can you say how Thunernetes compares to Agones for running game servers on K8S?

https://agones.dev/site/

see their udp proxy. so you do not make ip public, but proxy these.

Not really. Minecraft server notoriously ran in JVM for most of it's existence. Container virtualization is unlikely to add more than a few microseconds of latency.

This is my field, so even though I’m literally titled “expert”[0]- take what I say with a pinch of salt.

When it comes to gameserver performance what tends to be the biggest issue is predictable performance. So, kubernetes is not inherently antithetical to running game servers as long as you don’t share the cpu core(s) with anything.

In fact there is a project that (while I was not a part of) was spearheaded by ubisoft in collaboration with Google for running game servers on Kubernetes (called Agones)[1].

Ubisoft also used “Thunderhead” (which this project seems to be a variant of) on Azure with rainbow 6 siege. So it’s definitely not considered a show-stopping problem.

When it comes to containers/VMs the biggest hit in terms of performance is in this order: disk/network/memory access/CPU ;; when talking about game servers the most important things go in the reverse of that order, which means that the biggest performance losses are not as important. Network latency might seem like a big deal but you’re limited to frame times in the best case and geographical differentiations will eat much more than the combined weight of frames and Kubernetes networking.

Agones itself bypasses a lot of the Kubernetes networking latency additions, though, not all.

[0]: https://www.linkedin.com/in/jharasym

[1]: https://agones.dev/site/

Using container technology doesn't necessarily mean adding virtual network devices/proxying/forwarding. In practical terms, you can run in "host" networking mode (no separate network namespace) on Docker or Kubernetes, and most people would still consider those "containers".

In this case they seem to be NAT'ing packets from a host port on the correct Kubernetes node (the one running the container) to a port in the container, which can be done fast enough with iptables (or similar mechanism used by Kubernetes).

Not really. The major concern is getting clients to one machine and running a stateful connection so you don't need to pass messages through some kind of slow, multi-hop broadcast system. The CPU overhead of containers or vms is pretty minimal compared to this main concern.

Not necessarily in my experience! Most games don't hit the CPU particularly hard.

Mostly reliant on (hardware offloaded) network latency, memory, and storage. Consistency being the thing of most importance. A spike is felt - interpolation can only go so far

CPU virtualization extensions make the latency cost of VMs hardly noticeable, containers are technically better but it's a split hair.

On an unloaded host and a single VM you can achieve essentially identical performance as bare metal - the CPU extensions, pinning cores, and huge pages are key

Some performance metrics even improve! Disk operations tend to do better from an added layer of memory involvement

Edit: there are options for multi-tenant fairness whichever way you go -- Bare metal, VMs, or containers

Performance isn't much of a worry, more so the 'handles'

Nothing about kubernetes implies virtualization. People only use VMs because they're easier to provision. Technically you could provision physical servers with kubernetes and run containers on them with zero overhead.

Or perhaps more realistic you could provision a tier of very low latency VMs.

These days it feels like the real value of kubernetes is not the ability to "planet scale" but actually the control-plane API. With tools like kind, it actually is really easy to get going locally, no cloud needed.

I see no reason to not use it, even if your deployment cluster lives on a single vm. The API is worth it.

delivering your service/app via kubernetes is slowly becoming the standard way, just like an universal binary format. so as a developer you should build towards delivery for kubernetes. but you are certainly right that one should avoid running your own kubernetes infrastructure - at least until you can afford the accompanying expenses

I guess it makes sense for Microsoft to be able to put Xbox game servers in Azure but also game devs can benefit from the scalability of Kubernetes. If have is low tire not paying for hardware you aren't using and vice versa.

The vast majority of game servers are hosted on public cloud providers. They turn out to be a pretty perfect example of an ideal cloud workload:

- Very elastic demand curve

- Generally ephemeral with any necessary persistent state stored off-host

- Benefit from geographic distribution of cloud regions because they're generally quite latency sensitive

It might be more expensive to run but you have savings in terms of time to deployment, orchestration and the overall maturity of the cloud itself.

Managing bare metal at scale is challenging.

You can always run Kubernetes on bare metal. This seems easy in this case because you don't need persistent storage or load balancing to use Thundernetes, and you gain the ability to use the generic Kubernetes tools for self-healing, scaling, rolling deployments, etc.

Would like to know your 2cents :)