OpenFreeMap survived 100k requests per second

I'm sure there is some botting, it's basically guaranteed, but I wouldn't be surprised if nearly half the traffic was "legitimate". The bots don't normally need to reload (or even load) the map tiles anyways.

I managed a few Hetzner cloud instances, and some report perfect uptime for over a year. The ones that don't, I was the root cause.

What exactly leads you to make this sort of claim? Do you actually have any data or are you just running your mouth off?

That's how agreements work. If someone says they will sell a hamburger for $5, and another person pays $5 for a hamburger, then they are entitled to a hamburger.

>On a free service.

It's up to the owner to price the service. Being overwhelmed by traffic when there are no limits is not a problem limited only to free services.

> Financially, the plan is to keep renting servers until they cover the bandwidth. I believe it can be self-sustainable if enough people subscribe to the support plans.

Especially since he said Cloudflare is providing the CDN for free... Yes, running the origins costs money, but in most cases, default fd limits are low, and you can push them a lot higher. At some point you'll run into i/o limits, but I think the I/O at the origin seems pretty managable if my napkin math was right.

If the files are all tiny, and the fd limit is the actual bottleneck, there's ways to make that work better too. IMHO, it doesn't make sense to accept a inbound connection if you can't get a fd to read a file for it, so better to limit the concurrent connections and let connections sit in the listen queue and have a short keepalive time out to make sure you're not wasting your fds on idle connections. With no other knowledge, I'd put the connection limit at half the FD limit, assuming the origin server is dedicated for this and serves static files exclusively. But, to be honest, if I set up something like this, I probably wouldn't have thought about FD limits until they got hit, so no big deal ... hopefully whatever I used to monitor would include available fds by default and I'd have noticed, but it's not a default output everywhere.

The block was done based on URL origin rather than client/token, why wouldn't a rate limiter solution consider the same? For this case (a site which uses the API) it would work perfectly fine. Especially since the bots don't even care about the information from this API so non-site based bots aren't even going to bother to pull the OpenFreeMap tiles.

> Also, the servers were doing 200 Mbps, so I couldn't have kept up _much_ longer, no matter the limits.

For cost reasons or system overload?

If system overload ... What kind of storage? Are you monitoring disk i/o? What kind of CPU do you have in your system? I used to push almost 10GBps with https on dual E5-2690 [2], but it was a larger file. 2690s were high end, but something more modern will have much better AES acceleration and should do better than 200 Mbps almost regardless of what it is.

[1] to be honest, I'm not sure I understand the intent of open_file_cache... Opening files is usually not that expensive; maybe at hundreds of thousands of rps or if you have a very complex filesystem. PS don't put tens of thousands of files in a directory. Everything works better if you take your ten thousand files and put one hundred files into each of one hundred directories. You can experiment to see what works best with your load, but a tree where you've got N layers of M directories and the last layer has M files is a good plan, 64 <= M <= 256. The goal is keeping the directories compact so searching and editing is effective.

[2] https://www.intel.com/content/www/us/en/products/sku/64596/i...

NVMe disks are incredibly fast and 1k rps is not a lot (IIRC my n100 seems to be capable of ~40k if not for the 1 Gbit NIC bottlenecking). I'd try benchmarking without the tuning options you've got. Like do you actually get 40k concurrent connections from cloudflare? If you have connections to your upstream kept alive (so no constant slow starts), ideally you have numCores workers and they each do one thing at a time, and that's enough to max out your NIC. You only add concurrency if latency prevents you from maxing bandwidth.

Why would that kind of rate cause a problem over time?

But it depends on the project architecture. If the tiles are needed only client-side, then there's really little reason to cache things on _my_ server. That would imply I'm better at caching openstreetmap tiles than... openstreetmap. Plus you're just making the system needlessly more complicated.

And there's little reason for openstreet map to be upset, since it's not like _I_ am making 2 million requests - there are 2 million separate users of osm+. It's aligned to their incentive of having more people use osm and derived works. All is well.

Seriously, whose first thought on reading that is “oh great, I can exploit this”.

You are not talking about an insane amount of data if it's 56 Gbit/s. Of course a caching server could handle that.

Source: Has written servers that saturated 40gig (with TLS) on an old quadcore.

Presumably a caching server would be 10GbE, 40GbE, or 100GbE

56Gbit/sec of pre-generated data is definitely something that you can handle from 1 or 2 decent servers, assuming each request doesn't generate a huge number of random disk reads or something

cloudflare would be pay per request, a hefty sum if ddos happens

> Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that".