Ask HN: How much traffic to expect if your project hits HN front page? | Hacker News Reader

Ask HN: How much traffic to expect if your project hits HN front page?

53 pointswillismichael43 comments

I have no idea what the population of HN is, not the global distribution thereof. I'm working on a pet project that I would like to show of at some point, and in the event that it actually hits the front page (unlikely as it may be), I would like to know if I have the budget to spin up enough servers to handle the load, or if I should just point people at the github repo.

When other people have had their project show up on the front page, is there any pattern of how many concurrent users you topped out at, and how long most of them stuck around?

View on HN

I submitted a blog post a while back that made it to the front page and it hit about 11,000 visitors from HN and ancillary HN feed sites alone in 6-8 hours. The traffic was elevated for the next couple weeks and even my website, which was barely linked from the blog, saw a 50-60% increase in traffic lasting about a week after the post. I still saw traffic from this spike up to a full month later.

My total hits on / of when PDFy hit the frontpage (https://news.ycombinator.com/item?id=8034431), and remained there for some 24 hours, iirc:

    root@debian:/var/log/lighttpd/pdf.cryto.net# cat access.log | grep "GET / " | grep "news.ycombinator.com" | wc -l
    19387

That said, the Hacker News post led to a bunch of other places writing about it a day or so later, the most notable of which was Gigazine:

    root@debian:/var/log/lighttpd/pdf.cryto.net# cat access.log | grep "GET / " | grep "gigazine.net" | wc -l
    544

But most of their traffic came from the document viewer they embedded in their article:

    root@debian:/var/log/lighttpd/pdf.cryto.net# cat access.log | grep "GET /d/C8gHjDOxTLdunq1a/embed" | grep "gigazine.net" | wc -l
    41609

And this is what the bandwidth usage looked like during those few days:

    root@debian:/var/log/lighttpd/pdf.cryto.net# vnstat -d

    eth0  /  daily

         day         rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
      [...]
      07/13/14    767.19 MiB |  949.01 MiB |    1.68 GiB |  162.72 kbit/s
      07/14/14      1.27 GiB |    9.13 GiB |   10.40 GiB |    1.01 Mbit/s
      07/15/14      7.05 GiB |  106.73 GiB |  113.79 GiB |   11.05 Mbit/s
      07/16/14      2.89 GiB |   48.73 GiB |   51.62 GiB |    5.01 Mbit/s
      07/17/14      2.22 GiB |   24.21 GiB |   26.43 GiB |    2.57 Mbit/s
      07/18/14      1.23 GiB |   11.90 GiB |   13.13 GiB |    1.27 Mbit/s
      07/19/14      1.31 GiB |   11.88 GiB |   13.19 GiB |    1.28 Mbit/s
      07/20/14      1.38 GiB |    7.73 GiB |    9.11 GiB |  884.50 kbit/s
      07/21/14      1.44 GiB |    9.55 GiB |   10.99 GiB |    1.07 Mbit/s
      [...]

If I recall correctly, my HTTPd was hit with some 50-100 reqs/sec total (for static + dynamic). It didn't really have any issues with it, despite running on a cheap VPS with 512MB of RAM, on a non-optimal stack (lighttpd + PHP + MySQL).

I've noticed a significant increase of recurring traffic since (it still hovers at about 5-15GB of traffic a day as opposed to the 2GB before, and there's a steady stream of uploads).

As long as you don't run something obscenely heavy like WordPress or Joomla, and you don't use Apache, you'll probably be fine.

Do you mind to mention the VPS hosting company? (you mentioned it is cheap: https://news.ycombinator.com/item?id=8039666)

It's RamNode (http://ramnode.com/) :)

I got a DDoS-mitigated VPS in Seattle. I believe the plan I have is normally $15, but I used a coupon code so I pay $9.30 recurring.

I can definitely recommend them - however, I should add that their DDoS mitigation appears to suffer from the same issues as all other cheap VPS DDoS mitigation proxies; speeds are not always reliable, and connections occasionally break halfway through. That's not a problem with RamNode though, but with the mitigation provider (CNServers in this case) and/or proxy setup - their own connectivity is rock solid.

Some other hosts I can recommend in a similar vein are RAM Host (http://ramhost.us/) and VPS-Forge (http://vps-forge.com/), in case you want to set up a redundant system of sorts. I've hosted with both for years, and they're both rock solid and very helpful as well. (Relatively) small operations like RamNode, but very reliable.

Thanks for sharing the stats! Are you still operating PDFy on a 512MB RAM VPS? Did you have to do any optimizations to the web server config to handle the traffic?

Yup, it's still running on the same VPS. I do plan on offloading PDF file serving eventually, to a 'CDN' of sorts that I'm working on, but that's more for storage space reasons (especially cost-wise).

The only real 'optimization' was this classic one:

    server.max-fds = 2048

If you forget that, you're going to have a very bad time when you get hit by a serious traffic surge :)

The full configuration is here: https://gist.github.com/joepie91/e5bd63710b5910d2287a

It's really just a mostly standard config, some things pieced together. PHP is configured in on-demand mode, though - iirc, the default PHP configuration that ships with lighttpd on Debian is not.

I had a project that was number 3 on the front page for a good part of a Saturday.

I had 7500 unique visitors from HN including the traffic coming from linkbots that re-serve HN links.

With a single node.js (express) app on an EC2 medium instance and I was fine. I got about 10% conversion rate. It was a game with a signup page that required you to register first.

The single instance held up the static content and the app itself for a while. In hindsight I should have used an nginx reverse proxy for the homepage.

--EDIT-- here is the post: https://news.ycombinator.com/item?id=7364927

--EDIT2-- changed conversion rate typo from 1% to 10%

So by "1% conversion rate" you mean that around 75 people actually played your game? I see some discussion on your thread about how your server got hammered - was it the people hitting the landing page, or in-game requests?

OOPS! I meant 10% of 7500!

About 750 people signed up to play and there were about 500 games played (a game needs exactly 2 players).

The homepage stayed up the whole time. The stuff that got hurt was some actual gameplay due to an exception that was getting thrown, and I hotfixed.

Ha! Nothing like recruiting a whole swarm of HNers to find some obscure bug :)

I'm encouraged by your metrics of about 750 people and about 500 games - I think I could probably afford enough compute power to support that for a limited amount of time.

I've gotten as much as 60,000 in 24 hours if it stays on the front page. Recently I had one that hit 4000 in about 20 minutes but some circuit breaker hit (which I have no clue about, not really controversial or anything) and it dropped instantly to page 5 so the traffic died quickly. Generally at the top you might see as much as 10/s peak. I've seen 500+ concurrent but of course that is based on whatever Google considers concurrent.

Never had any trouble serving it (my own code, LAMP, on Amazon micro instance). programmer.reddit.com is a little lighter. Ancillary traffic (other sites) from a front page post might add 10% or so over time.

> I've seen 500+ concurrent but of course that is based on whatever Google considers concurrent.

Almost sure that it means at least one hit within the last 5 minutes.

My website usually has very little activity (about 100-200 visits per day for an open source project) and out of nowhere I went #1 on HN last week-end for a blog article.

From saturday to monday: 80,935 unique visits with peaks of 600 simultaneous people on site. Out of those 81K, 24,661 came from HN. The average time spent on site was 2:11. No real pattern, things started going viral as soon as the article hit the front page (which took a couple of hours). Things died off very quickly after 3 days of intense load.

Now I had never expected that type of load... In fact my blog is hosted on the cheapest shared hosting service NearlyFreeSpeech. I want to mention that they held the charge perfectly. I wrote a quick article about it with more numbers: http://www.nicolasbize.com/blog/and-the-best-shared-hosting-...

I submitted a pet-project called ngProgress[0], a progressbar provider for AngularJS. It's not a project per se, more a smaller library I decided to share here. It was on the frontpage for almost one day and on the second page for a day as well. I got about 10'000 visits during the first day and about half, 5000 during the second day. This also includes shares on Twitter that came after posting it here.

Most people just opened the page and closed it within ten seconds. The second largest group had the page opened for about one minute before closing. Please note the landing page for ngProgress[1] is very simple though and has almost no engagement except demo for the library.

[0] - https://news.ycombinator.com/item?id=6250112

[1] - http://victorbjelkholm.github.io/ngProgress/

The link to my data visualization was on front page for ~8 hours. I got 10K hits first day, 2500 second day, 400 third day. The post was not specifically pimping my side project, but was a visualization of Seed Funding. That being said we had about 500 people from HN request beta access.

link: https://www.machete.io/board/view/seed_db_funding_rounds/157...

If you want to track visits from HN, MAKE SURE YOU ENABLE HTTPS BY DEFAULT AND LINK TO AN HTTPS LINK. It is in the http spec that no referrer info is passes from an https site (like HN) to an http site.

One last bit - we are hosted on a small Azure instance, we used loader.io to test what kind of a load we could handle and it shit out pretty quickly. We implemented some output caching and it handled the HN flood just fine (200-300 concurrent users).

My post [1] made it to #8 on the front page last week and stayed there about 5 hours. I haven't made proper analysis yet, but here are rough numbers. The page got ~3500 visits, most of which within the first 24 hours. Most concurrent users was ~80. The first 24 hours got the linked app ~500 downloads, so that would give a pretty good conversion rate. I don't have data on how much of that came through the website or HN, though.

[1] https://news.ycombinator.com/item?id=8070131

When my Xerox Story was hit last year (http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...) I got about 200k hits in two days, at peaks reaching about 10k per hour. All numbers are from google analytics so real values might be higher. Also, it's not only hn, consider also the follow-ups (in my case, shortly after hn, the front pages of slashdot and reddit hit me as well).

Addition: I use Dokuwiki as a blogging system, which was accompanied by a varnish cache. The combination of both was able to cope with the load (and additional load from mass media taking up the story) very well on a standard hosting server.

Apparently, more than GrandArmy can handle with this USPS redesign post...

I also saw ~7000 visits first day but that was in 2012. Here are some conclusions of my web traffic analysis:

https://github.com/entaroadun/hnpickup/wiki/Hacker-News-Pick...

Here are multiple screen shots of the google web traffic analytics interface:

http://hnpickup.appspot.com/hnpickup_web_app_statistics_snap...

My WordPress blogs have hit the HN front page a coupla times.

I got about 6000 hits in an hour the first time and 10,000 the second time.

WP-SuperCache coped admirably in both cases. The mod_rewrite caching was enough to cope with HN. (I sent the developer, Donncha O Caoimh, £10 with gratitude!)

But what really made the server cry: being on HN led directly to being on Reddit, where the second popular post got 80,000 hits in a day. In this case I had to put WP SuperCache into direct-cache mode. Then it was fine.

A blog article of mine (hosted on Squarespace) was on the top of the frontpage for most of a day back in January and received approx 25,000 pageviews.

(And also, incidentally, 42 article comments and counting, without a single nasty/sarky/snarky one.)

The same article has since received 500 Facebook likes and was tweeted around 360 times.

https://news.ycombinator.com/item?id=7075537 | 334 points

This time last year my project VimSnake.com was in the top 3 for most of the day. I made the stats public:

http://statcounter.com/p9177631/summary/daily-rpu-labels-bar...

https://news.ycombinator.com/item?id=6223946

We've had a couple of blog posts hit #1. From memory, we received around 35,000 visitors on each occasion, spaced over the course of several hours.

What was your converstion percentage, and what kind of hardware (virtual or otherwise) did you run to support this?

Yes, I had pretty much have the same numbers.

I had this Android app https://play.google.com/store/apps/details?id=com.hour.chat on the front page last week. It got 45 upvotes and also around 450, pretty good.

So for an app you could probably say 10 downloads per upvote.

Been on the frontpage a few times, mostly weekends. 2/3 days got me 30,000 to 60,000 hits/day (24h) per time. My website is static, optimized and hosted on a cluster of geo-spread nodes so they were still doing nothing, the statistics server (piwik) did have a load of 4 at peak...

When a few readers collectively had the poor judgement to vote a post I wrote to the front page it brought in around 12k uniques from HN and associated parasitic sites over the space of 6 hours or so. This was peaking around 200 - 250 simultaneous visitors.

Unless it goes viral outside of HN, based on my experience of a few front page hits, you can get 10K-50K visitors. As most of visitors will bounce, serve a static landing page and you can handle the traffic easily with a single server

My site made it several times to HN main page. Rule of thump is, that every upvote equals about 100 visitors, while the site is on HN, and that about same number of visitors come during the next days by tweets and facebooks.

I got 26k pageviews when I managed to reach first page (http://blog.talles.me/my-hacker-news-front-page-day.html).

Happened to me a few times (for technical articles/code). If I recall correctly it was around 3000-7000 unique visitors. My site is hosted on Github pages so I didn't have to worry about the load.

I've had a few posts hit the front page over the last couple of years. I had between 5,000 and 10,000 unique visitors (as Google Analytics defines them) over the following 24hrs for each post.

10-20k total if it doesn't go viral outside of HN. Maybe 100-200 concurrent at peak. Don't worry about your server, unless it's a box at your home it should handle the traffic...

I've got my project on the front page #20, it led to 500 visitors in 3 hours. Not a huge success i would say :) The interesting question is how much _conversion_ to expect.

My blog http://jquer.in was briefly (about 3 hours) on the front page. I got around 1500 Unique Visitors according to GA.

Deleted comment

Are you sure about the numbers? That'd be millions of users.

If I recall correctly, I seemed to top out around 300 visitors per minute. Load stayed close to that for about 2 days (dunno about US evening hours)

About 10 to 20,000 in my experience for a site that's on the front page for about a day.

I agree ~10,000 visits for a standard front page story, with ~200 simultaneous visitors.

I got ~26k from 12 hours as number 2.

Reply on news.ycombinator.com