A link preview with an image is over 100 MB? That sounds insane. And if they mean the total traffic in 5 minutes was 100MB that cannot possibly be bringing Cloudflare to its knees. That is an indictment of Cloudflare’s CDN then!
I think what’s being implied here is that, when you share a link to Facebook, Facebook will access the page to generate a link preview, so will download a tiny bit of HTML and an image. But when you share a link on mastodon, that link immediately gets propagated to many other mastodon servers, which then propagate it to others, so suddenly many thousands of mastodon instances are simultaneously downloading a little bit of HTML and an image, and the cumulative effect of that in this instance was 100MB over a minute or two.
It does seem like a typical static website ought to not have a problem serving that, especially if it’s behind Cloudflare. It seems odd that a single EC2 instance would have a hard time serving that.
But given more than one person is complaining about, it also seems like each mastodon instance could very easily delay propagation of the story by a few minutes to soften the blow here.
I liked that idea at first glance, but thinking about it, CDN performance would actually be better with a single huge burst than if they were smeared out (assuming a very short max-age so the site can be updated rapidly).
In theory a CDN could optimistically coalesce requests then re-send them when the headers of the first one return. But this is very complex and rarely done in practice.
This can also occur on any time the cache gets stale and needs to be refetched.
I don't think this is true. It certainly isn't for any CDN that I've worked for or on.
Cloudflare don't do this either - they use a cache lock - the first request basically acts as a blocker for all the others, leaving the other requests waiting for the response (if it's cacheable they serve that response, if not then they proceed to origin).
It's normally configurable, but most sane CDNs do have it enabled by default, precisely because big bursts tend to be sharp in nature and a cache miss can be origin breaking at that point.
Just for completeness's sake, Nginx's HTTP proxy module can do it too (the setting's proxy_cache_lock) though it is off by default there.
What does getting a certificate through let's encrypt have to do with the server getting overwhelemed?
It's a thing that happens once to renew a cert every couple months.
The performance hit of https on a modern server is negligable. The performance hit on a watch is negligable.
Cloudflare is either passing traffic through without touching it, or as a proxy is doing tls termination, as they're a trusted CA in most devices/browsers/OSs/etc.
None of this really has anything to do with that's happening with OP.
All the issues sound more like a them problem than mastadon or cloudflare.
Something's wrong with their setup. Hard to tell as now HN has brought them to their knees
Agreed, and hardly surprising is it that if the site can't cope with link preview traffic that non-trivial page views would be trouble too!
ActivityPub itself could more intelligently cache or display the content but something doesn’t add up if Cloudflare can’t handle that kind of traffic let alone Hacker News traffic.
From the article:
> Presently, we use Cloudflare as our CDN or WAF, as it is a widely adopted solution.
To me, it sounds like the author isn't really familiar with the difference between a CDN and WAF is; or familiar with Cloudflare beyond it being a popular thing they should probably have for that matter.
The images arn't the problem, it's their dynamicly generated page, even if the content is pretty static.
But I agree this does seem strange, like it shouldn't be unmanageable load at all.