Hacker News Reader

Comment by jrochkind1 | Hacker News Reader

All of github pages was run off of _one_ server? (with one failover standby).

That's pretty amazing. If all you're serving is static assets, apparently you have to grow to pretty huge scale before one server will not be sufficient.

I'm curious if there was at least a caching layer, so every request didn't hit the SSD. They didn't mention it.

View on HN

We do have Fastly in front of *.github.io, but there's still a significant amount of traffic (on the order of thousands of requests per second throughout the day) that make it through to our own infrastructure.

We don't do any other caching on our own, although the other replies are correct in that the Linux kernel has its own filesystem cache which means not all requests end up hitting the SSD.

What percentage of the requests is handled by CDN?

I believe that only CNAMEs use the CDN for custom domains (naked domains do not) [0]. So that rules out a chunk but there will be lots of cache misses too. Maybe someone from GH can confirm.

[0] https://help.github.com/articles/about-custom-domains-for-gi...

github.io will still also hit the CDN. It's just custom naked domains that will miss.

Their docs mention ever 200 response code request is cached, so I'd say a majority of the requests?

Are the assets extracted from git and dumped on the filesystem or are they fetched from the Git objects? aka If I generate a git tree object with millions of files, but only pointing to a handful of blobs (think a maze) would this exploded on the filesystem?

Do you explicitly invalidate Fastly, or do you use a timeout? If so, how long is the timeout?

Invalidating quickly is the main problem with caching on a CDN. We use an async worker via Redis pubsub to call for expiration on individual files locally on proxy servers we run. I'm looking at using NSQ for this in the future. One interesting solution is to just use an HTTP hit to expire a cache, which you can see a flavor of in our nginx config file. Nginx in effect becomes it's own cache SoA. We needed a special nginx module to make that work, default nginx only lets you expire the entire cache.

Fastly probably has something similar. You've really got to do this within 5 seconds or your user is going to get pissed off waiting for it every time they save/reload the page.

The big problem with passing caching to third party CDNs is that they need to be able to handle your SSL certs inline to request static files you don't need to change the URL of (because then you would be changing the published content which you don't control), and supports wildcards.

In effect you're doing what Cloudflare does. I'd use Cloudflare in a minute for this, but wildcards require their commercial plan and we can't afford it ($6k/mo). Also we would need fine-grained cache expire.

If Fastly does this and is priced in our range, maybe we should talk to them. :)

FWIW Fastly invalidates objects, services, and batches of objects globally at ~150ms via an API. They have a different approach to wildcard/batch invalidation: it groups based on a prior-set header that can be programmatically based off of URL paths (or a bunch of other things).

Disclaimer: I'm employed by Fastly :)

That would work for Neocities. Wildcard pricing is unfortunately a little out of our budget, but if you were interested in coming to an arrangement with us (we could say powered by Fastly on the site, for example) in exchange for a reduction on that, let me know. :) contact@neocities.org

I believe this is automatically done by the operating system. Nginx usually performs some kind of mmap of the requested file, and the virtual memory management will automatically take care of caching.

BTW, I'm often surprised how people are afraid from opening files from their scripts, as if they think this will always lead to disk access. Then, they start implementing a hand-written caching layer on top of that, which usually performs worse than what the OS already offers.

If it's a static file with no dynamic compression (nginx allows the use of pre-compressed assets) or SSL, nginx doesn't even have to open the file, it just uses the sendfile system call.

Minor nit. The sendfile system call requires an open file. nginx can just skip reading the file contents into userspace.

Also, BSD has some recent extensions to make sendfile work with ssl [https://people.freebsd.org/~rrs/asiabsd_2015_tls.pdf] (pdf)

Correct. Static file serving is amazingly efficient. Even the filesystem caches for you automagically with whatever RAM you're not using. There's even a kernel function for speeding up file transfers, that's how optimized this stuff is: http://man7.org/linux/man-pages/man2/sendfile.2.html

My rough estimates are that Neocities can handle 20-50 million sites with only two fileservers and a sharding strategy. So double it for each shard, and you've got a solution that actually scales pretty well.

It's not that surprising. Serving static files is pretty easy and lightweight (part of why I like the movement towards SPAs), and there's been 20+ years of work done to make it fast. We've been conditioned for years by dynamic application servers that perform multiple DB queries, external service calls, etc per request. It doesn't have to be slow or inefficient.

SSD cache and os file system cache is always there. I'm sure they were filled with ram, and that's all it would be used for.

Reply on news.ycombinator.com