A Census of Minecraft Servers | Hacker News Reader

A Census of Minecraft Servers

blog.bithole.dev

245 pointssidedishes75 comments

One thing the author may have missed here because I didn’t see it mentioned: this also misses any servers not hosted on the default port of 25565. I don’t know how common this really is, but Minecraft has SRV record support, and for those of us running more than one Minecraft server on a single box it’s likely at least one will be on a different port.

It’s very common for cheaper Minecraft SaaS hosts to serve it on a non-default port.

> Minecraft has SRV record support

He’s scanning IP addresses, not dns names, so there’s no easy way to get SRV records. He could first do a reverse DNS lookup, but that would slow things down tremendously and also there are many, many Minecraft servers running without DNS names

His point is that because of SRV record, using the default port is not needed, thus it's make it more likely that a server wouldn't mind being on another port.

Yeah, I usually set up private servers to use some meme port and I'm sure that's not an uncommon practice

25565 is less common than alternate ports from what I've seen. Most small, individually run, servers are not running 25565.

So a proper census would do a reverse dns lookup on every IP looking for a "minecraft" SRV record and connecting to that.

That won’t get them all either, and likewise will only find one server per IP. The only really reliable way would be to scan every port to see if there’s a Minecraft server there, but that blows up the search space by quite a bit.

Reverse DNS lookups only return one designated name for the IP, not all records referencing the IP.

Probably not common but definitely non-zero.

I'm in roughly the same boat as you (minus the multiple instances). My personal Minecraft server I run on a machine sitting behind me at home for friends uses SRV records. I bump the port number each "major" version release (yeah, I know, "minor" version in semver semantics) in case I ever have to run a previous version for whatever reason. This gets reflected in the SRV records. I've never actually had a reason to do this, but it's there if I need to.

Or at least that's the theory. I haven't updated it for 1.18. Not sure I will unless I run Tekkit or something in the future.

1.18 is gorgeous. You're missing out

Extremely common on managed server providers, not so much on self-hosting. So, I think a vast amount is missing under the iceberg.

Our Minecraft server is spun up only when we play, with a simple bespoke front-end that allows any Cognito-authenticated user to start or stop the AWS instance. The dashboard also shows an auto-updated leaderboard with some in-game stats (death count, miles traveled, last login and so on), stored in DynamoDB and periodically updated by a cron job on the server. I also planned to add a map of the spawn, but not sure if that’ll happen.

We have not played since MS started effectively requiring a phone number from every Minecraft player.

I'd love to read a blog post or a link to a repo to see how this is done. Sounds very fun and interesting!

Thanks for the interest, I might publish the thing after auditing and documenting. There are hard-coded insensitive credentials like Cognito pool ID and player UUIDs, which I should make nicely configurable. If I do it soon enough I’ll reply here. It’s very basic-looking though (and without dynamic registration, intended for small infrequently changing groups with memberships managed by an admin).

As I have already been using AWS and this was not at all business-critical, I did not care about vendor lock-in and thought of it as an exercise in how much I could delegate (neither letting random visitors access our dashboard, nor spending time implementing custom auth). Their JS SDK documentation wasn’t great, but after some digging it was somewhat straightforward to make a fully static SPA (hosted on S3) access specified AWS resources (per IAM policy) on behalf of authenticated Cognito user.

The app also “integrates” with Discord to ping a channel on each instance start/stop, but that is merely posting to a webhook URL.

I wrote it in React and TypeScript with a bare-minimal Babel + Webpack configuration but it could just as well be written in vanilla JS.

As to the server, it is plain Ubuntu with a cron job that periodically tries to launch MC server if it’s not already running (or something silly like that). Another cron job publishes stats to DynamoDB (IAM policy allowing the instance access the table), and I wanted to add yet another job to generate a PNG with a pretty map of the spawn.

I've got a similar setup myself, check out https://lobste.rs/s/yppnts/tailscale_on_nixos_new_minecraft_... where I describe how I did it. It's quite custom but all open source. https://github.com/infinisil/on-demand-minecraft is the main part

Fancy! At first look it’s a no-brainer to go with, so either our setup existed before yours was public or I did a shoddy job at researching options.

Part of my idea was an actual web home for the server (leaderboard, map, etc.) so if we play MC again I will see if that is possible with your project.

What's the latency like with that? When a user causes a wake event, how long until the server is live and ready to play? What sort of wait system do you use? This sounds really cool.

In my setup it is instance start + cron job timer + MC server initialization. AWS instance wakes up pretty quickly, cron job is configurable (I set it to 15 seconds or so), MC may depend on the size of the world and any mods I suppose (we played vanilla). All in all took between 40–100 seconds I think.

I... kinda can't believe that one of my libraries (Forgelin) is the second most common mod on all Minecraft servers. It doesn't support the five most recent versions of the game and I haven't touched it in years. I guess there are a lot of other mods (or a few really popular ones) for somewhat older versions that use Kotlin.

All online games have factions that think their game has been downhill since one version or another and stay there with private servers. Minecraft has a few such factions, so your plugin probably serves one of them.

It's super common to run older server versions due to mod support. Unless there's some new mechanic in an update that people want to use, not using the latest version isn't really a big deal. Easiest to grab the mods you like and just freeze it.

That part doesn't surprise me, it's just the sheer magnitude of 41% of modded servers that's shocking.

I’m kind of amazed that you can still do massive scans like this and not face some bad consequences like getting blocked somewhere, or having your hosting provider get very annoyed at you.

1 packet going out to 4 billion hosts probably doesn't even matter next to a few 4K video streams.

I expect if author had launched it through CGNAT, someone wouldn't have been pleased. (Correct me if I'm wrong)

Presumably CGNAT implementations have some safeguards to prevent one client from exhausting state tables.

Your host isn't likely to care unless it causes them grief. There are so many scans happening at any given moment I doubt any of the targets will notice yours.

I do vaguely remember one of these, "I scanned the whole internet! It's easy!" stories from years ago where the author wound up receiving an email from someone at Electric Boat who told them, "Please don't portscan us. We're required to call the FBI when it happens." Your host would probably be "very annoyed" if they received one of those, but I doubt anyone cares enough to send such a message nowadays.

>I scanned the whole internet!

This one? http://census2012.sourceforge.net/paper.html

My favorite part is the animated day/night cycle at the end.

Portscan them more so they fix their process.

I don't know if the tool is called Massscan (I thought there is a tool called something with rabbit) but I followed a security blog a few years back and since then I had the impression that scanning all ipv4s is basically a solved issue.

The most interesting part of this to me was the list of top mods by popularity. But when I looked a lot of them seemed for much older Minecraft versions or for old versions of the mods. For instance "Pam's HarvestCraft" is mentioned but that's been deprecated in favor of a HarvestCraft 2.

Is that just a quirk of how the mod names are reported and folks are really running newer stuff? Are older modded servers still popular? Are the servers themselves mostly old and no longer used?

Many people still run on older modded versions. To use the example you gave, HarvestCraft 2's CurseForge page[0] says:

> Welcome to the brand new HarvestCraft for 1.14.4 and beyond! Please read carefully as this is NOT a update of Pam's HarvestCraft but a re-boot.

1.14.4 is 3 years old, but many (many!) servers are still running on 1.12 or older versions if doing modded. There simply isn't an incentive for many mod owners to update their mods to the latest version, so the "community updates" as a whole are generally quite slow and people end up stuck on their favourite version.

Edit: Looking at the article, they also only analyze Forge mods. On newer versions, other mod loaders are gaining popularity, whereas Forge is the de-facto mod loader for 1.12.

[0] https://www.curseforge.com/minecraft/mc-mods/pams-harvestcra...

> There simply isn't an incentive for many mod owners to update their mods to the latest version

There is quite a strong incentive. What there is not, is means — Forge and Minecraft have both changed dramatically between versions, to the point that many mod developers throw their hands up in the air and rewrite the mod entirely. For something complex enough that that isn't an option, for instance Electrical Age, it's easy to remain stuck on an older version forever.

There's no documentation, and the API owners often assume that forcing a complete rewrite of major parts of the mod is fine. It's really not.

Some mods are also forever stuck at a given version as the source isn’t open and the mod author doesn’t want to port, is missing, or has died.

Thaumcraft 4 is a big example.

It's a miracle that the Minecraft mod scene exists at all. The old MCP project that made tools to decompile the obfuscated code to something readable and then recompile it again was just the most remarkable hack. I guess that's years out of date now and I vaguely recall Microsoft was more open to supporting mod authors in recent years. But I guess from this discussion it's still kind of a mess.

How sad is it that Microsoft is more open to supporting game mods than Mojang when Mojang promised a modding API was just around the corner for years and owed a lot of their success to the modders they seemed to spite.

The funny thing is that before the buyout (and maybe after, I don't know) Mojang's DRM was an abysmal failure. They'd obfuscate their Java binaries in a vain effort to prevent cracks but every cracked version out there would just patch the launcher and run the game unmodified. Cracked launchers would run updated versions just fine and when Mojang actually tried to check for a cracked launcher within the game itself they had a rash of false positives triggering on actual paying customers and nothing but a minor road bump for the pirates. The only thing obfuscating the game binaries did was force MCP to put in a mountain of effort to building the tooling and mappings to deobfuscate it so that mods could be developed. The entire MCP project should not have even been necessary. All this for some crappy attempt at obfuscating their binaries that never even worked against pirates in the first place.

At least a silver lining is they finally have data packs and command blocks.

Additionally, as far as I know, after 1.12, the development of minecraft paced up and a lot of internals of minecraft changed. This turns the update of mods past 1.12 into a complete rewrite most of the times, which has burned out quite a few authors.

This is correct yeah, there's a lot of effort involved in porting mods to newer versions. Larger mods slowly move in that direction, but at the end of the day even now people are playing modpacks from 1.12 because there simply aren't equivalent ones in newer versions (with the same set / feel of mods)

To throw in another reason why old versions are still popular, a good chunk of modded players use modpacks, and some of the more popular ones are pretty highly polished collections of mods with a bunch of glue to make them work together as a somewhat unified experience. This can't really happen until the mod ecosystem for a given version has already stabilised a fair amount (and can only happen on versions for which a large number of mods are available, which tends to be every 5 releases or so).

Some of them could be Modpack servers. There are only certain versions with support for most mods (1.12.2, 1.7.10, sometimes 1.10). Many of the more established modpacks still run on 1.7.10 or even older

What a pleasant surprise, I know the author from somewhere else. Small world!

Very enjoyable read and even more interesting results. 4 Minecraft servers per 10000 people in Germany is kind of insane to think about.

What was the number 2? The author made it sound like it was the US, but that has 4x less

The subtle advertising:

> This is probably thanks to cheap hosting offerings from companies like Hetzner (insert link).

Conflicts with their own data:

- OVH 24,417

- QuadraNet 9,927

- GPORTAL 9,339

- GMO Internet 5,466

- Hetzner 5,327

For those not in the game server hosting scene, it's because OVH offers a robust DDOS system for free for their servers. The price is not much different either. OVH is by far the leader for ALL game servers due to DDOS protection, price, and value. Hetzner is good, but OVH is a league of it's own.

Are you in the game server hosting scene? Curious how you came by such an insight

I used to run a few big private game servers, yes.

You have to take the quote in context:

> However, Germany ends up taking the prize for most Minecraft servers per capita, with a whopping four servers for every 10,000 people. This is probably thanks to cheap hosting offerings from companies like Hetzner.

Hetzner is a German company, OVH is not. And while OVH has a presence in Germany, if we run this query on Shodan: https://www.shodan.io/search?query=product%3A%22Minecraft%22... OVH is #8 while Hetzner is #2. So I'm not sure why you'd think this is advertising, they're merely providing an example for Germany.

That being said, GPortal is also a German company and #3 overall, but they're a dedicated provider of game servers, rather than commodity servers. Conspicuously absent is Nitrado - also a German company - probably because they run most of their servers on non-default ports, whereas GPortal assigns each server its own IPv4 last I checked.

> Hetzner is a German company, OVH is not.

> That being said, GPortal is also a German company and #3 overall,

Gportal (Ociris GmbH) is #1 in Germany, by far. 8,291 (24.2%) vs 3,380 (9.8%)

> but they're a dedicated provider of game servers, rather than commodity servers.

Why does that matter? The statement is about Minecraft servers and why Germany is popular.

Seems if you were going to name drop one to support that reasoning you'd use the most popular one.

Heztner is neither the leading Germany Minecraft provider, nor the leading overall provider, both by a wide margin.

I appreciate you joining HN to comment and clear that up though.

Why they chose that example I can only speculate, perhaps it was the first name they recognised and just used as an example because they were familiar with it?

I just really didn't like the assertion that this was supposed to be some sort of advert, rather than an arbitrary pick. It certainly isn't an affiliate link or anything.

> I just really didn't like the assertion that this was supposed to be some sort of advert, rather than an arbitrary pick. It certainly isn't an affiliate link or anything.

Feel free to speculate one way or another, I'll do the same based on the data and reasoning provided.

You don't need an affiliate link to advertise btw, referer headers do just fine.

I really wasn't slamming them for advertising, just noting it was and that it didn't fit their reasoning as to why they were plugging them.

If you feel it was an arbitrary selection, that's fine. I was just expanding on the statement as to why it doesn't fit the data. I assume you concede to that right?

What's up with MineTest? Seems like the only issue is some of the mods aren't maintained, but other than that, it looks like there's some real potential.

It would be interesting to see a similar census, and if it's gaining any ground.

I tried it a while ago and unfortunately immediately ran into some rather serious bugs.

My kids have been using it on a regular basis for a couple of years now, the core engine is stable and I didn't find any bug, and there are nice games and non-buggy mods.

I briefly played with it on a Pi4 just to see what the fuss is about(Partly because I wanted to figure out why they chose to include proprietary minecraft instead since its supposed to be hacker friendly).

Seems fine except there are no creepers in in any of the apt-gettable mods I can find, so that might get in the way of preinstalled it fully ready to play.

I sure do admire the patience it takes to build anything in either. It almost seems like a good fit for a history themed game that teaches how hard things were to bootstrap, a la Oregon Trail almost.

"Fun" fact: Microsoft set up Minecraft to be open to everyone by default. Consequence: griefing groups that scan the web and blow the ever living shit of whatever they can find on those random servers (TNT blocks can be supplied by item duplication exploits): https://youtu.be/hoS0PM20KJk

Yup, my family minecraft server fell victim to that. Thankfully it was a new map with only a couple days' work into it, but annoying nonetheless.

Whitelist your private servers!

Requiring a VPN is how I handled that - not so much for preventing people from joining the game but because I don’t like open ports.

I’m surprised that VPN usage isn’t more common for minecraft players.

It's not more common because it doesn't really make sense.

Can you explain this a bit further?

Is white listing users the way to avoid it? I’m possibly too cautious and only have a couple of ports allowed though.

In essence, without knowing much more about your setup, you're just shifting responsibility somewhere else.

Whitelisting is the route most (no actual statistics to backup, but based on personal intuition) would take because the server can prevent a connection after making a authentication (online-mode=true) and authorization (white-list=true) check.

Mesh like VPN solutions used to be popular in the past with software like Hamachi but, at least in my experience, performance was dismal and required additional setup for potentially non-technically minded end users.

Really good post. It's amazing how fast you can scan all of IPv4.

I also needed tune SQLite recently for an event logging server, and "PRAGMA journal_mode=WAL" helped a lot.

Yes, the author would probably get another huge speed up there. Some more potential optimizations here [1].

[1] https://phiresky.github.io/blog/2020/sqlite-performance-tuni...

> Really good post. It's amazing how fast you can scan all of IPv4.

If you're just scanning one port, which the author seems to have done, you can probably do it in some minute or two, unless you wanna play nice and lower the rate of sending packets.

Otherwise if you wanna scan full IPv4 + all ports, it'll take a couple of minutes at least. Masscan with the right hardware seems to be able to do it in five minutes or less.

speaking of massscan, how about those dang expanse palo alto networks guys.

> YES IM STILL SITTING IN THE SAME SPOT I WAS THE LAST TIME YOU SCANNED ME AN HOUR AGO.

I'm surprised that the majority of servers are running an unmodified version of the server software. I had expected the majority of servers to be running a Bukkit-compatible modification at this point.

I guess I should add a "Blog Posts" page to wiki.vg for good quality dev-orientated articles like this.

https://wiki.vg/Dev-focused_Blog_Articles

This article is wonderful, I love the working knowledge. A lot of the idea feels similar to the recent Tom7 Harder Drive video: https://youtu.be/JcJSW7Rprio

I had never seen some of this IP charting and stuff and in his video he does a lot of similar stuff.

It depends on what server you want to play. It is not hard to download a pirated version of Minecraft and play it without paying. But, if you do it, you won't have access to most online servers available on the internet. In my opinion, the best thing to do is to buy it. After that, you can download any server you want from https://servers-minecraft.net/minecraft-bedwars-servers . Check it out by yourself. I am sure you won't be disappointed about it. Also, you can build your server here.

> how many Minecraft servers are out there? Drumroll, please...

160,992

On IPv4, running on port 25565, which were online at the time of being scanned

… and were publicly accessible.

and were Java servers. Bedrock not included (runs on a different port over UDP only). Probably not including any Java or Bedrock Realms (official MSFT servers), who likely seat all access behind an authenticated gateway of some sort.

For extra fun, is a wiki.vg page trying to accumulate documentation on the Bedrock UDP interface. https://wiki.vg/Bedrock_Protocol

This is still a neat sample of Java Minecraft servers.

Absolutely.

I wonder how many servers are completely hidden. I’d love an easier way of making a server that wasn’t accessible to the world at large.

It isn’t so much that setting up the VPN is painful, but helping a kid install the software over the phone with no tech-savvy parent around is hard.

a few dozen transactions a second seems horrendously slow no matter what is going on under the hood there.

Do ISP block this sort of mass scanning? Seems like something that is easy to detect and block.

No, why would they? It's like pissing in the ocean.

Reply on news.ycombinator.com