Hacker News Reader

Comment by majewsky | Hacker News Reader

> ability to handle HN hug-of-death

Seriously, this is bullshit. When my blog was on the frontpage, CPU load on my single 1-core VPS peaked at 5% and bandwidth usage was about 1 megabit per second. No Netlify, no Cloudflare, no nothing. If your website cannot handle the HN crowd, it's just badly engineered. Period.

Source: https://xyrillian.de/thoughts/posts/latency-matters-aftermat...

View on HN

Yeah: I honestly don't understand how people build websites that can't handle Hacker News levels of traffic... this site just isn't _that_ popular in the grand scheme of things.

The instant your website requires a database query to build the front page you're susceptible to the HN hug of death. Obviously you can build a scalable solution to handle traffic spikes, but there will always be the danger.

If you're talking about an ordinary webserver giving out an ordinary webpage you will run out of bandwidth long before you overload the server.

Yeah; static pages like blog posts should really be cached out from templates + data, to avoid this problem.

Depends on your database. S3 will handle the hug just fine.

That's a bit dramatic. I've load-tested database backed sites on $5/month DO droplets and gotten > 1k requests per second for extended tests.

My primary site not only stores articles in the DB, but runs them through an extended, custom markdown parser upon retrieval, and processes and reports event data on every page load. It was still successfully returning ~250 reqs/sec before I'd bothered implementing any caching or pre-compilation of markdown at all.

In contrast, people recently HNed[1] are reporting about 12-15k visits over a day. That's something a single server on a $5/month VPS can handle in a minute

1) https://getpolarized.io/2019/04/02/getting-hacker-news-lesso...

> In contrast, people recently HNed[1] are reporting about 12-15k visits over a day.

This matches my experience. I think I got about 10k visits over the course of a day, 80% within the first three hours.

A quick check shows the site is on DreamHost which offers $2.59 a month shared hosting which isn't going to handle the load (though they have the full assortment of hosting options you'd see, up to dedicated hardware).

I wonder how much variance there is in traffic for sites that are on the front-page. That variance could be based on a combination of the hour of the day (because of the varying timezones of HN readers), the day of the week (we don't all check every single day), and the particular topic of a post (I would think that the average individual doesn't click on everything). It could be that while your site was on the front-page, others at other times, other days, or of other topics could have received visits many times more (or less) than what you received.

Worthy of a submission of its own.

Same for me, I have a static HTML site and it never even broke a sweat.

A blog typically comes with a comment function. That's something dynamic which requires lookup in some sort of database. Your page is purely static and does not even have pagination between posts. Hosting that is trivial and nothing to boast about. Your comparison is really unfair and needlessly nasty.

A comment function doesn't necessarily require a database lookup for every request, you can cache the blog page for a second or two so that it doesn't collapse when getting a hundred requests per second.

Yes and that makes things magnitudes more complex than just serving a static HTML page.

Not really, it's just basic configuration of nginx or something like that. Putting up nginx with microcaching in front of your webserver is literally a five minute task. If you have no idea on how to do it, then it's going to take half an hour with some tutorials, so even in that case it's not that much more work than serving a static HTML page.

Oh how cool is that, I never even heard that term before. Thanks! It does look quite complex though, I have to learn about proxying and nginx.

Displaying comments doesn't have to be dynamic. Most need reviewing anyway. https://github.com/eduardoboucas/staticman

If you're doing a database lookup for every page load for comments, you're doing something wrong. Also, you'll be moderating comments, unless you want your blog to become a cesspool of hatred or spam, so the page becomes even less dynamic. Myself, I'd put the comments on a separate resource so that the static page would never be affected by that load, whether POSTing or GETing, and load async. (so your static site is always up, even if your comment system falls over)

You are not doing something wrong, you are simply not designing for high-load. Loading things dynamically on every request is perfectly fine for almost any blog.

Moderation is offtopic. Putting comments on a different system seems like overzealous, premature optimisation...

I would expect open connections to be the bottleneck (not CPU or mem). Do you just have very low latency and thus relatively few concurrent open connections (yes, I saw the graphs in your link, but it's hard to figure out how many concurrent connections are open at a time)?

EDIT: Downvoters, would really like to know why you're downvoting? Am I being naive in thinking a blog would be bottlenecked on connections?

My site has been on the front page a few times.

Concurrent requests were usually in the 400-800 range depending on where it ranked and what it was about. That's going by Google Analytics too which means it's likely higher since devs tend to disable as much tracking as possible.

nginx won't even break a sweat if you're on a low end VPS and are serving static content. You'll have pretty much the same latency as serving 1 concurrent request (single digit milliseconds + whatever network latency there is between your server and the visitor).

I have once handled a website being HNed and /.ed a few years ago. The issue turned out to be the default Apache config from distro, which was very ill-suited for such traffic, generating tons of processes which struggled to run on a small VPS. Switching to nginx fixed it; I suppose switching Apache to mpm_event model (or even just tuning the threading configuration) would be enough too.

Or low total bandwidth. A personal blog not expecting a lot of traffic may only have puddle of transfer bandwidth per month. Depending on the content of the post, that can max out a cap rather quickly.

I don’t understand the downvotes here.

A small blog can randomly get a 100,000 view spike. If the blog contains images and maybe even a gif it’s easy to blow through bandwidth caps for most free tiers.

I use netlify’s free tier but actually host my videos on BunnyCDN. I also make sure to optimize my pngs and jpgs. This isn't precautionary. It's actually necessary.

Reply on news.ycombinator.com