Hacker News Reader

Comment by maxpert | Hacker News Reader

Ok I am confused, 60k hits in a day? What broght down the website? 72MB of size, network congestion, or 60k hits? Even with authentication for download what can bring the system down? I have handled more traffic on a RPi with 100MBps connection. I really don’t get it.

View on HN

Here I made a quick copy and a quick domain, almost 10 LoC and it will be more reliable than what Cambridge had http://hawking.sibte.ml/

Wow! Thanks. But, I do not understand the <script> tag in homepage contains lots of ![]+[].

I think it's just a way to document.write his email address, so no bot/crawler can directly see it.

Edit: It evaluate to `document.write('HIS_EMAIL at gmail')`

If the browser can do it what's stoping a headless browser bot from evaluating it all the same? Is it just to lower the attack surface?

>If the browser can do it what's stoping a headless browser bot from evaluating it all the same?

The vast majority of crawlers aren't that smart, headless browsers used to be painfully slow so no one used them.

Evaluating all javascript to see if it eventually writes an email address is not a good idea.

In fact doing so probably violates the halting problem somehow.

Love the irony on that! Which makes me wonder how much valuable content and research in locked away on technologically ancient servers in Cambridge and Oxford.

More modern and up to date than you'd think :) Some departments here in Oxford have a server life-cycle of 3-5 years. It's just that nobody bothers for higher than expected volume of traffic unfortunately (a practice that can be extrapolated to many things in academia).

(60000 connections * 72 MB) / (24 hours * 40 Mbps) ~= 10 concurrent connections.

If your server is configured for a low number of concurrent connections, it can easily seem swamped if a small number of people are concurrently downloading a large file slowly. All that's happening is that it's not accept()ing new connections until existing ones finish.

Heh, I don't know what the current hardware is for the CUDL DSpace repository, but having helped upgrade it a few years ago, I can't say the software is the fastest.

It's a java application with XLST in the page rendering pipeline, so not exactly optimised for speed.

If you're hitting a single box, this seems perfectly possible - especially if the server or the network is poorly configured. This is what CDNs are for.

This isn't Joe's blog running from a desktop found in a dumpster. It's a university with actual infrastructure. They should be able to handle this. Registration is probably more load.

A 72MB file being served 500,000 times over a 24hr period is 3-4Gb/sec.

The department probably has one or more web servers serving content off a network drive. Your average "rack of network storage" won't even blink at that.

There was probably a gigabit switch somewhere or that was being a bottleneck or the web server was simply misconfigured for this task.

+1, thanks for the correction. Apparently I have a skewed perspective only working on web at big corporate joints.

Probably a burst of traffic hitting a single box web server, rather than 60k hits spread out evenly over a day.

With a half time of 3 days of interest and a 100 MBps connection the average download time would be 3 hours. That would indeed seem unusual for today. but 30k page hits spaced out over 60 hours is not too much of a burden on top of that.

It hit top of reddit yesterday, I don't think it was a mere 60k hits.

More than 400k, actually: https://mobile.twitter.com/dannykay68/status/922745216481808...

Tweeted her :P let's see if she is bothered to respond

  What brought down the website?

The Greys.

Reply on news.ycombinator.com