Hacker News Reader

Comment by oreally | Hacker News Reader

> with the availability of Cloudflare, running high-traffic websites is much more cost effective.

sidetrack but how does cloudflare make things cost effective? wouldn't it be cheaper if i just hosted the wiki on a simple vps?

View on HN

More than a decade has passed since then so I am stretching my memory. At peak we were serving in the region of 10 million page views per day which made us one of the most popular websites on the internet (Minecraft was a phenomenon and every Minecraft player needed the wiki). We were probably the highest traffic Wiki after Wikipedia. Nowadays Cloudflare could absorb most traffic because of the highly cacheable nature of it, but at the time, Cloudflare didn't exist, and every request hit our servers.

Yeah, Wikia in aggregate was in the top 50, maybe a top 20 site at various points. Wikia was built on caching. From my memory, about 99% of page views hit some kind of cache. If that dropped down to 97%, servers started to suffer. It's good to remember that the Fastly CDN company is a spinoff of Wikia, it was developed internally there first. Without that (varnish cache plus lots of memcache) Wikia would not have been able to handle the traffic. Mediawiki is horribly inefficient and one reason why Wikia was attractive as a host was that we had figured out a bunch of tricks to run it efficiently. The default configuration of mediawiki/wikipedia is real bad. Bigger independent wikis just couldn't handle the scale and many of the best independent wikis moved there for that reason. Just as one example, every link/url on a page hits a hook/callback that can call into an extension literally anywhere in the code base, which was several million lines of PHP code. I remember the "Batman" page on the DC wiki used to take several minutes to render a new copy if it fell out of the cache. That was one page I used for performance optimization tests. The muppet wiki and the lyrics wiki also had huge performance issues and fixing them was some of the most fun engineering work I've done. Every useful feature had some kind of horrible performance side effect, so it was always a fun puzzle. I also hate landing on a Fandom wiki now but thanks to the actual editors, it's still got some good content.

How much of your server load was Grand Exchange Market Watch?

I bet, being by some counts the most popular video game ever - but which also makes it kind of a bad example to use when talking about wikis.

By definition, very few wikis will have to deal with becoming one of the most popular websites. (And as you say, at that point one should be able to figure out funding.)

Cloudflare get the best deals on bandwidth. It will usually be cheaper to serve a terabyte from Cloudflare than to do it yourself: you could probably run the wiki on the free plan!

Perhaps, but VPS traffic prices are also already a lot better than "big cloud" traffic prices, especially if you choose your VPS provider with that in mind. And once your traffic is large enough there are also options where you pay for a fixed pipe instead of a transfer amount.

> Cloudflare get the best deals on bandwidth.

If you want to pay for bandwidth then yeah, CloudFlare is a great option.

Otherwise, if you like the experience of not paying per GB/TB, go for a dedicated server with unmetered connection that has the same price every month, regardless.

Cloudflare don't charge per GB/TB. You get unlimited bandwidth even on their free plan. The problem with paying per GB is that it's in the CDN's interest for you to get a DDOS attack so they can charge you for all the bandwidth. It's in Cloudflare's interest to reduce DDOS attacks and unwanted bot traffic because it costs them bandwidth, not you.

Your point on interest is spot on.

I moved a few of my personal websites to AWS's CloudFront and it cost me like a buck a month, way cheaper than maintaining a virtual server to do it. Except that somebody somewhere decided to try their DDOS tool on one of them for a few hours in the middle of the night, and I got a bill for $2541.69.

Eventually they credited it, but it was not a fun ride, and decided that I was done using a CDN with misaligned incentives: https://sfba.social/@williampietri/111687143220465824

> it's in the CDN's interest for you to get a DDOS

What kind of conspiracy is this? As if anyone charging for bandwidth hopes to get their infrastructure attacked

The whole point of systemic incentives is that there is no conspiracy. Nobody wants a DDOS and every large provider will have people genuinely working to avoid them. But every time there is an opportunity to allocate resources, the team that gets to frame their return on investment in terms of real dollars will always have an edge over one whose value is realized only in murky customer satisfaction projections. Over the lifetime of a company, the impact of these decisions will add up with no need for any of the individuals involved to even be aware of the dynamic, much less conspire to perpetuate it.

And then you have someone like the founder of Fly.io who has been explicit about that mindset at least once:

> putting work into features specifically to minimize how much people spend seems like a good way to fail a company

Found the source for more context: https://news.ycombinator.com/item?id=24699292

That's sound logic. In this specific case of capitalistic incentives, I haven't noticed that it's working out in a way that make one more vulnerable to DDoS when one pays for bandwidth

Why not? They have the capacity they could absorb nearly any kind of attack without blinking.

You don't need to pay anything to run TBs through Cloudflare, you could use the free plan.

Rent VPS or managed hosting or host wherever you want, proxy it with Cloudflare on the free plan, Cloudflare caches it.

It's more like: if you have a website that (sometimes) gets a lot of traffic, do you want Cloudflare to cache it and serve it with very few hits to your cheap server, or do you want your compute costs to expand to cope with the requests?

> do you want Cloudflare to cache it and serve it with very few hits to your cheap server, or do you want your compute costs to expand to cope with the requests?

Usually you have something like a platform/tool/service that is mostly static requests that could be cached, with some dynamic requests that couldn't, as they're CRUD requests or similar.

If your struggling to serve static content, then do go ahead and slap Cloudflare on top of that bad boy and probably your visitors will be a bit happier, instead of upgrading from a cheap VPS.

If you're struggling to serve the dynamic requests, Cloudflare/CDN won't matter because these things actually need to be processed by your backend.

So instead of trying to shave 50ms off from my simple static requests with a CDN, I'd much happier to optimize for all the requests, including the "dynamic requests" that need to hit the backend anyway.

I'll still go for a dedicated server with proper connection and performance rather than a shitty cheap VPS with a CDN in front off it.

> If you want to pay for bandwidth then yeah, CloudFlare is a great option.

The free plan is a lot bigger than you think.

> dedicated server with unmetered connection

And where have you found one of those with reasonable pricing?

Hetzner is pretty cheap, but only offers Europe location for their dedicated servers last time I checked. For more locations, DataPacket is nice, although a bit more expensive.

If you can run your application on Cloudflare Pages / Workers with Cloudflare's storage/DB things, it really gets dirt cheap (if not free) and very fast. And even without that, Cloudflare's caching CDN is very good, very cheap and very easy.

Ten years ago bandwidth was expensive. Still is, even if not as much. A simple VPS gets overwhelmed, but a simple VPS behind cloudflare can do quite well.

    s/cloudflare/a CDN/

Cloudflare caches pages at many many datacenters, often colocated with large ISPs.

This lets Cloudflare deliver pages from their local cache over local links (which is fast and cheap), instead of fetching the data every time across the world from wherever the VPS is located.

Reply on news.ycombinator.com