Why HN Got Slow | Hacker News Reader

Why HN Got Slow

152 pointspg58 comments

We got MSMed. The post about baseball bat sales in the UK was the perfect media bait and they all linked to the item page.

I turned on comment thread pagination to decrease the load. I'll turn it off tonight.

View on HN

Forgive my lack of worldliness but what is "MSMed"?

Mainstream media'd. Meaning someone from the mainstream media posted a link to that post, and the surge in traffic made everything slow.

Main-stream media.

I was searching on google trying to figure this out myself...

Slashdotted Digged Reddited

Pick one

They aren't considered MSM.

NYT, Slate, Guardian, etc., rather.

It was a historical reference

2nded, I haven't heard 'Slashdotted' to mean getting traffic from slashdot.org in a while, but it does seem to have stuck as the means of saying we got a ton of traffic from some other website linking to us.

Your point is pretty reasonable, considering the meme sprang to popular awareness in it's 'Slashdotted' form, doubtless melting many servers down prior to appearing in the MSM.

We got MSMed. The post about baseball bat sales in the UK was the perfect media bait and they all linked to the item page.

Hmm. The past 48 hours a lot of sites have been slow for me: HN, Salon, FARK, ...

I figured I was getting Cox'd. http://cox.com

Cable providers provide terrible DNS service. You may notice a big difference in performance by changing your DNS settings to http://code.google.com/speed/public-dns/ or http://www.opendns.com/. As a bonus, I've noticed that many service interruptions are DNS failures, not routing issues.

You might want to try NameBench, which will find the fastest DNS server for you.

http://code.google.com/p/namebench/

Be Careful about changing your DNS settings if you use a CDN - iTunes, in particular, offers horrible performance if you use googles DNS.

Why do you think iTunes would care what DNS server you are using? I'm genuinely curious, as I would think that the DNS lookups are abstracted by the OS, and even if they were not, I cannot see how using a different (esp. a better performing) DNS server would hinder iTunes performance — especially since the original DNS isn't provided by Apple or anything.

CDNs often rely on your DNS info to determine which node is closest to you and, thus, which node should offer the best performance.

Using a central DNS like Google's can do two things:

1) Break locality. Using 8.8.8.8 may cause CDNs to potentially direct you to a node that's nowhere near your actual physical location

2) Aggregate traffic on a node. The more people using 8.8.8.8 for DNS, the more people who may land on the CDN nodes associated with 8.8.8.8's location and thus that node may be more loaded than the one closer to you

[edit] Google even covers this in their Public DNS FAQ: http://code.google.com/speed/public-dns/faq.html#cdn. They offer 31 different locations, while Akamai may offer thousands of nodes (I can't find hard numbers, but they refer to 1,200+ points of presence here http://www.akamai.com/hdonline and 1,000 networks here http://www.akamai.com/html/technology/edgeplatform.html). You're basically constraining yourself to a small subset of at least Akamai's CDN by using Google Public DNS.

That completely slipped my mind. You are, of course, correct.

An alternative suggestion here is to use Comcast's 75.75.75.75 IP address; while it may seem static, it's actually an anycast address, and will resolve to the nearest node.

One thing many people forget when choosing a DNS server is that it's not just the ping time that counts - the time it takes the server to reply to the actual DNS request is what matters.

e.g. while 75.75.75.75 for me (Comcast Chicago) is a 15ms trip and 8.8.8.8 is a 28ms ping, the lookup on the former (dig x.com @dns_ip) can reach 120ms while on Google's DNS it never exceeds ~35ms.

> CDNs often rely on your DNS info to determine which node is closest to you and, thus, which node should offer the best performance.

Why do they do this? Wasn't DNS SUPPOSED to be something that wasn't in any way tied to locality? Are CDN's just using the side effect/fact of life that they happened to fall out that way, because it's easier?

Pretty much, yes. But it's not just easier, it's also simpler. If you do the distribution at the DNS level (which is pretty much as low as you can go), you don't have to deal with the much messier geo-distribution on a higher level.

Absolutely - change the DNS _and_ put something like DD-WRT on your router with dnsmasq as the local caching proxy server. Nothing beats that per Steve Gibson's DNS benchmark!

Was this an inside joke?[1]

1. http://allthatiswrong.wordpress.com/2009/10/11/steve-gibson-...

2. http://news.ycombinator.com/item?id=2510614

It wasn't - this is just a DNS benchmark that does DNS resolution in loop and measures how long it took etc. Nothing requiring security expertise.

Besides I don't understand why few people are after Gibson - he is generally on the mark - that's not to say he doesn't make mistakes - but nothing that makes me want to totally ignore him.

Thanks, but I actually do that using both Google's (8.8.8.8), as well as Level 3's (4.2.2.1)

Debug the connection with http://netalyzr.icsi.berkeley.edu/

Thank you.

I was wondering why the load times have been ridiculously long.

Also: I've been getting Cox'd as well the past three or so days. Maybe I should complain, or something.

fyi, cox has their own local speed tests which can be informative as to where the hangup might be. The local one to me is:

http://azspeedtest.cox.com/

It's actually somewhat frustrating to see that register phenomenal speeds when the web past their internal routers seems ... buffering ... buffering.

I figured I was getting Cox'd

That's what she said...

I noticed the same with Fark as well. Also HuffPo.

Will we have International Haskell Day later this week?

You know, I still have a 50-odd Erlang article queue. So I'm not saying I want every day to be Haskell Day, but I'm also not saying I'd be terribly upset if I had a few hundred articles on Haskell to keep me warm at night...

What do you use for this queue? I can't possibly keep up with all the reading I come across, and often just relegate it to Readability, Instapaper, Delicious, Xmarks, etc. Would be very interested to hear how someone else handles this.

I have a historical Erlang article http://news.ycombinator.com/item?id=2866337

Now there's a joke for the old-timers.

Those front pages filled with articles on relatively obscure programming languages are actually what got me to start reading HN.

PG, can you do the complimentary blog post "Well, these are my statistics to HN before and after getting MSM'd"

Link to the article?

nm, here it is: http://www.pcworld.com/article/237616/baseball_bat_sales_ris...

Crazy that a site such as PCWorld could bog down a site as popular (or so it seemed) as Hacker News. Goes to show what kind of a small fish Hacker News is in a much bigger pond..

Well, I'd be surprised too if it was just due to PCWorld, but it's more than that. The guardian and the telegraph, and many other mainstream news outlets have gotten wind of this story: http://www.google.com/search?client=opera&rls=en&q=b...

Ah, I was using Yahoo! Site Explorer as a judge.. which only showed PCWorld. http://siteexplorer.search.yahoo.com/search;_ylt=A0oGk.jo20F...

I think you missed the point. What happened, was everyone (actual news organizations) linked to HN because the top link had a "scandalous" title. This wasn't because of a link from PCWorld

I'd be curious to see some numbers. Just for fun, and also to get an idea of the amount of visitors that kind of link can bring.

Capacity planning, they say :)

Thanks PG, you must wear many hats.

What ever it is, it's always a white one. ("Yeaaaaah")

I was looking at [a list of hats](http://en.wikipedia.org/wiki/Hat) recently, by the way.

Looks like HN has now some focus of link spammers.. see recent submissions.

Can caching help?

Interesting that that post itself (http://news.ycombinator.com/item?id=2864557) is now on the 2nd page despite having 400some upvotes in the past 7 hours (odd because there are several posts with many less votes in a similar timeframe on the homepage).

Does that happen due to being flagged too many times or some other reason?

Speculation: there is an authority metric which is used as a strong quality indicator. If you're considered an authority then your vote counts a lot more.

7 votes from folks like patio11 counts for about a million votes by the likes of me.

Like I said, speculation.

I'm pretty sure it has to do with flagging. (that, as well, is speculation combined with personal observation).

Would norvig be on the list? aaronsw? I keep meaning to build a list of celebrity CS people on HN. Anybody already got one?

This would be a good start: http://news.ycombinator.com/leaders

What amazes me is that as much as I read HN, there are still people in the top 10 whose comments and submissions I've never seen. I wonder if they got all their karma years ago and it's taking this long for everyone else to catch up.

To be fair, some of them mostly submit stories (compare e.g. tptacek and fogus; the first has lots of comments but few stories, the other is the other way round.)

That's almost exactly what I'm not interested in. There are some very interesting people with account who post very infrequently, but their comments, in the context of who they are, is very interesting indeed.

true to the word. " What would you believe , maximus?"

community tagging could not be the solution here- what do you think?

--Stackoverflow blog offers some explanation.

So...we're Digg then?

Yikes...THAT wasn't the right to say

I'm wondering also if some who down vote get a bigger down vote ratio. usually i lose points 1 by 1 and sometimes i just reload and lose 10 or more. Or that's cache. ;-)

And yeah I've strong opinions so while sometimes i get crazy upvotes I also get crazy downvotes hehe.

There's no down voting on articles. And for karma, 1 vote = 1 point.

Any time to address the endless number of reposts and the increase in volume of submissions?

I'd like to see accounts that only submit the same site over and over again and don't otherwise participate penalized. That'd clear up the 'new' list a lot as well as dampen the duplicate submissions.

Reply on news.ycombinator.com