High-Throughput Game Message Server with Python WebSockets | Hacker News Reader

High-Throughput Game Message Server with Python WebSockets

mortoray.com

68 pointsingve56 comments

Looks like a perfect case for erlang. And while we are at it, one of existing xmpp servers written in erlang. One puny $2/month server will serve thousands of users without sweating.

I feel like I have to ask: where are you getting a server that'll run arbitrary erlang code for 2 bucks a month? The absolute cheapest I've seen seems to be 5, from digitalocean/linode

There are a few websites which aggregate VPS offers, for example https://lowendbox.com/ or https://lowendstock.com/

However, you usually get what you pay for, so be prepared for

    - long and frequent server outages
    - tech support which takes weeks to respond or no tech support at all
    - IP changes without warning
    - system images which have last been updated 5 years ago
    - machines which simply fail to reboot
    - retroactive contract changes, e.g. introduction of a "maintenance fee" for already paid services (cloudatcost)
    - customer support logging in as root into your machine from Indian IP addresses and changing random system settings

But if you are not concerned about the privacy, integrity or availability of your services, you can often find offers for about $1 per month.

Fair warning you've posted, I'd say with a caveat some of the cheap ones are good enough.

There's also a community that [0] splintered off the ones you mentioned that are not ColoCrossing centric unlike LET/LEB. I'd bought hundreds of servers there over the years (with cost being a main factor) and it tended to be the Colocrossing ones that'd be volatile or not last your entire billing period.

For specific servers/recommendations, I have a few servers with Ramnode (might be a grandfathered service) for $15/y. There are a bunch of good providers in that price range but without knowledge of them they can be hit and miss and for the most part makes it worthwhile paying more to worry less about it.

[0] https://talk.lowendspirit.com/

I'm running a $2/month xmpp server on IONOS [1], so far didn't have a single outage.

However, they can slap 20% vat tax above that if you are in certain jurisdictions, making it $2.40 for EU customers.

[1]: https://www.ionos.com/servers/vps#packages

Scalewsy Stardust instances are 1.9Eur/m.

https://www.scaleway.com/en/stardust-instances/

IPv6 only is even cheaper at 0.3 euro per month.

Hetzner and a few others will go a bit cheaper than DigitalOcean. Scaleway used to but have hiked prices of their cheapest instance.

Vultr has $2.50/mo USD VPS if you're okay with ipv6-only... that's the lowest I've ever come across.

See IONOS, I provided a link in a reply to parent comment.

This is cool, thanks!

Author contemplating an eventual C++ rewrite, but this is a perfect use case for Rust. When you want performance and where memory mistakes can be a serious security problem.

Or anything that isn’t a scripting language

indeed, C#/Go/Java etc would all be a big boon without the mental overhead of Rust. Data races being caught at compile time is also nice though when doing a multithreaded server.

> but this is a perfect use case for Rust

Idk it seems like for someone prototyping things alone, Python is the perfect langauge

I don't think the parent is disagreeing with that, but the article mentions graduating from Python to something else, which is where the recommendation fits. Parent isn't saying the article's original version should be in Rust.

got it thx

Just wrote a websocket server in Rust. Turned out really easy.

Or java

I used to be on the java hate train, but honestly the JVM is magic for infra. I worked at AWS, and I groaned so hard when I started. Then, I came to appreciate it because of the tooling and the types of issues that actually cropped up.

This may also be stockholm syndrome.

the nice thing about Java is you get pretty good performance even while pretending the hardware concerns don't exist. This takes a lot of things to think about off your brain. The downside is you can't choose to think about them even if you want to.

Or F#/C#. Rust will have memory fragmentation over time.

Memory fragmentation is generally a problem for pets, not cattle. In 90% of use cases it’s nothing that frequently reaping processes won’t fix. This works regardless of language, as it pushes the job of garbage collection onto the os. Another nice feature of treating your processes like dirt is that you tend to get high fault tolerance and scalability as emergent properties.

Right. Just like the story of a memory leak in a missile.[1]

[1]: https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98...

I really think it's a good solution for many cases but I haven't seen any server-side web framework that implements a kill-the-process-often strategy.

Do you know of any? If so, when do they decide to kill the process and how do they handle client disconnect?

Some pre-forking web servers will shut down a child process after it has processed a configured number of requests.

E.g. Apache in mpm_prefork mode, MaxRequestsPerChild configuration.

C# suffers from that too. I've personally seen a prod instance experience OOM with something like 40% free virtual memory. There's a reason why ArrayPool is part of mscorlib.

All of that said, x64 greatly reduces the effects of memory fragmentation: there's always more virtual memory.

Previous discussion on this https://news.ycombinator.com/item?id=18190573

I prefer C or Go

Ah yes, C, well known for its memory safety!

As with any tool, if you don't use it properly it can hurt you.

That's quite a ridiculous statement in this context. Even highly-trained, extremely smart developers are proven to not be able to use C without introducing gaping security holes. That's not "it can hurt you". That's "it hurts you and everyone that is exposed to your RCE machine".

Have you looked at code by AntiRez, the author of Redis? Redis is written in plain C.

What about QMail and DJBDNS, from Dan Bernstein? Again, all C.

And the mother of all: the Linux kernel. Again, in C.

Redis has RCE exploits which would not exist in a memory safe language [1]. QMail had an RCE as well[2] and DJBDNS has had a security bug which would not have occurred in a memory safe languages [3]. The linux kernel has had an incredible amount of memory bugs over the years. The software you mentioned does much, much better then the average C written code. But it still has had issues with memory. So in fact all your examples prove my point.

[1] https://github.com/jas502n/Redis-RCE

[2] https://www.qualys.com/2020/05/19/cve-2005-1513/remote-code-...

[3] https://web.archive.org/web/20190126114638/http://article.gm...

C is not C++. There is a reason core services for DHCP, DNS, etc are built in C and not Rust. If you're a careless developer you can have security vulnerabilities in any language.

The reason they're built in C is legacy and the fact that Rust hasn't been around all that long.

Well, what's stopping you from rewriting it in Rust?

That's only true on Linux and other systems where the kernel has been written in C. On some platforms the kernel provides no backwards compatibility guarantees and that means backwards compatibility is handled by libc. This means C is enshrined into the platform and often the path of least resistance.

There was also a pure Rust Wayland binding that failed because Wayland assumes that you are using C semantics which made it hard to provide a safe interface for Rust that was also convenient.

Yes, and that good reason was that Rust did not exist.

I'd rather be hurt by my safety razor than by my chainsaw though.

I'm working on a game server handling websockets with mainly broadcast messages. I'm using nchan[1], a Nginx addon to handle websocket with a pub/sub pattern. It works really well, is customizable and is highly optimized. Perfect for some games which needs to send a global state to every players.

[1] https://nchan.io/

Have you considered trying Twisted? I recently watched this talk about how to use async to handle 1000s of concurrent players on a mobile MMO, it wasn't exactly enlightening but it was definitely worth watching: https://www.youtube.com/watch?v=KW3mzaM3UcU

I am currently planning a client server architecture for a game that requires Bullet Physics with UDP on the backend, and I'm weighing different options, I initially wanted to use Go but it would require interfacing Bullet's C++ API with C so it's not a feasible option. I'm currently considering Twisted + Bullet's Python API.

Nice write-up. While it's not appropriate for every game, UDP based networking may offer another order of magnitude improvement over (TCP-based) websockets. WebRTC can be used for unreliable datagrams.

I intend on adding WebRTC as a channel for higher volume discarble messages, such as the cursor position of all players.

Might as well plug the game server I wrote over the summer. It's a python UDP server that I was able to scale to over 100 concurrent users based on various sources, but mostly the Gaffer On Games blog.

I chose python because I wanted game development to be fun. Ultimately, I couldn't find an existing python framework for multiplayer games that scaled well to my needs.

[1] https://pypi.org/project/mpgameserver/

did you consider using AWS eventing in API Gateway https://aws.amazon.com/blogs/compute/announcing-websocket-ap... ?

I have not used AWS and I am not sure if I understand the pricing structure, but wouldn't it cost like $30,000 per month if the server is running at 12k messages/second?

https://aws.amazon.com/api-gateway/pricing/

At this point I feel like that most AWS services are for marketing purposes. The type of people who would pay 30k a month for 12k msg/s are very few.

It's just there so you can awe developers at the breath of services AWS provides.

Quesrions for people familiar with the industry: my understanding is that UDP is optimal for fast-pased games, you don't have to wait for re-sends of old packets, etc.

However, it seems you can't just flick a switch and get encrypted connection like you can with TLS.

So what do people normally use? Does everyone do a custom AES-packet or something?

Check out Valve's game network sockets layer, it addresses a lot of the things you're worried about

https://github.com/ValveSoftware/GameNetworkingSockets

Some games encrypt them, but while I would normally never say this - there's nothing really to protect over the udp packets if it's just game state. And you want to keep the latency as low as possible for those packets.

You want to do everything else over tcp and encrypted.

I remember quite a few aimbots back in the day would MITM packets so there are some reasons you might want to do it from a defense in depth perspective.

One option is DTLS[0], which is roughly TLS for UDP.

[0]: https://en.wikipedia.org/wiki/Datagram_Transport_Layer_Secur...

HTTP/3 with QUIC as transport will be using UDP and is encrypted.

I assume it will change quite a few things for these kind of needs.

If you want Python to go faster, use PyPy.

It'll probably help in this case since I suspect it's computational overhead introducing the bottleneck, nothing architectural.

Alas, I might have to wait, as I tend to use Python 3.8 and it's not supported yet -- though mainly I use it for types, it might actually run in 3.7

Reply on news.ycombinator.com