Hacker News Reader

Comment by sh1mmer | Hacker News Reader

Dear Pair Networks,

I'd like to highlight this lesson in how to loose (or not gain) customers by randomly shutting down technology sites that serve the decision makers you wish to influence.

Tom

P.S. Well done Slicehost.

View on HN

150K hits in 30 minutes is 83 hits/second. That's a lot to ask for from a shared-hosting account.

When the traffic starts to impair neighboring sites, something has to be done. Just about any ISP will do the same thing: block the site with the surge, that could possibly make other arrangements, rather than inconvenience other customers whose traffic is as expected/usual.

The detail missing so far is why Pair noticed today, if it was the same level of traffic as before, or a slow build. Was a new threshold crossed? (Did someone's HN-focused tool go haywire?)

The Pair message suggests end-of-day logs will be the way to tell for sure.

I was being tongue-in-cheek, but there is a real point here.

As you say I doubt today was unusual for HN, it's the way Pair went from zero to shut-down.

One of the things I like about the way that Joyent operate as a cloud host is that they allow you burst on shared boxes because of those times you need it. At the same time they'll let you know you need to think about buying more resources without just slamming on the brakes.

Pair should do a much better job of noticing a soft limit earlier, so if a heavy traffic day had hit HN PG et al should have been already aware they were overusing their shared hosting and were planning a route out.

That technology is not out there... A static site might be fun at that level and not causing problems, where as a WordPress site with super cache is fine, but then a Drupal site can only handle 1/4th of that.

It currently is not really possible to track that per user in a shared environment and see who is using what resources from an individual perspective of MySQL / Apache / CPU / Mem.

That technology is not out there

Ofcourse it is.

Or how do you think they determined that it was pg's site causing the trouble?

Absolutely wrong. That technology exists and is being used millions of times a day at sites all over the world. DreamHost, for example, employs just such a technology for their shared hosting: http://wiki.dreamhost.com/index.php/CPU_minutes

Exactly - they missed an obvious opportunity to upsell. If Trevor had gotten a mail stating that he needed to upgrade he would most probably just have done so and been happy. It's not like he can't afford it :-)

Yes and no.

I doubt it is a HN-Focused tool, as this affected the static content from www.ycombinator.com, and images and CSS are not the focus of bots usually.

Assuming it was not a DoS attack, a smart host should have noticed the traffic load increasing over time, and offered to upgrade to a less-loaded server and recommend a dedicated server.

By disabling the site, they have lost a customer, and lost on a up-sell to a dedicated server.

You're assuming the load today wasn't extraordinary. That's not yet clear. PG doesn't know of any surge, probably based on figures he has access to on other hosts and analytics tools. But what if HN/YC static resources at Pair were newly being deep-linked, by a much more popular site? Only seeing the end-of-day logs from Pair, compared to previous days, can definitively answer this question.

tlb says it was "gradual": http://news.ycombinator.com/item?id=1283439 .

I'm sure that's what the general News.YC/YC stats show. But as the Pair.com tech mentioned, today's logs only become available at midnight ET. If there was a surge this morning -- to the static resources only, as if by offsite deep-linking -- Pair's logs are the only place it would be evident.

So let's see... you're hosting a website that has gotten large; that is, it's grown to the point that it will need higher-cost services in order to meet demand. You have a chance to add a valuable customer to your client base. How best to handle this?

a) anything

b) except

c) killing their service

Exactly

How best to handle this?

Of course, the flip side is that leaving it running adopts an attitude of "screw all our other customers, they can eat crappy service while we kiss up to the popular guys who are chewing up everybody else's server resources". Which isn't what I'd look for in a hosting provider...

False dichotomy. The correct way to handle this would have been to temporarily move the shared server to hardware where it won't impact other customers and notify YC that they need to move their server to a bigger server or the site will have to be shut down. Presumably with an ultimatum of a week or whatever.

Agreed - the whole reason you pay a host is that there's some level of management and responsibility there. You're not just renting hardware.

This is exactly the kind of approach that a smart company which cares about business would take.

Companies like slicehost make their money on the customers that do not use their servers.

Yeah, Pair screwed themselves pretty badly by pulling this trick. They would have been far better off doing something like:

Dear [account_contact_user]

Your website traffic has risen beyond the maximum threshold of [threshold_amt] for the [name_of_level] level of service.

Since we appreciate your business of the last [length_of_service], we have given you a 24 hour courtesy upgrade to our next level of service -- [name_of_next_level]. If, by [end_time] you decide to keep this level of service you must contact our sales center to arrange payment. Otherwise we will have to start throttling traffic to your server so that it remains below the threshold of [threshold_amt] and does not impact our other customers.

If you have any questions about this courtesy upgrade, or wish to keep this new level of service, please contact [account_manager] at [account_manager_details].

Thank you for using Pair Networks for your hosting needs.

What about the dozen other customers who are calling you to complain about crappy service right now?

Todays's logs are similar to other days. I guess the traffic is high for a shared account, but I've never noticed it being slow so I didn't think about it.

I would have been happy to respond to an email saying I should upgrade & pay more, but even after a few emails with tech support that option didn't come up.

The only limit they advertise for that class of account is data transfer, and because it's mostly just serving uparrow.gif -type files we're well below that limit.

The "ask" is not that a shared hosting account support 83 hits/second. It's that the hosting company demonstrate the bare minimum of respect and professionalism in contacting the hostee ASAP when their site is being shut down due to excess traffic. This isn't bitbucket or geocities, dollars are changing hands. If you don't like running a business that makes money then by all means, treat your paying customers like they are a burden to you, they will get the hint and go elsewhere.

Customers nothing. "I didn't know what that 'hacker news' thing was or if it should have that much traffic so I shut it off". Really?

Please turn in your geek card at the door.

FWIW, It's probably not worth their effort of working out exactly what websites are using cheap shared hosting and overusing resources.

I don't get shared hosting at all. VPSs are dirt cheap these days.

VPSes are still shared hosting and are not immune from the problem.

But they are trivial to migrate between machines on demand.

They won't shut you down for over-using resources. As a matter of fact you can't over-use resources, you can only use the resources you are given to you. I know they are different type of VPS, I am referring to Slicehost/Linode type VPS which has hard limit and acts very much like dedicated single box.

More important to me is that I am really really really surprised that Ycombinator was running on a shared account. This just blew my mind.

I have no hard feelings towards pear, I would have shut the site down too; possibly long ago.

They limit memory at Slicehost but not CPU. I think you can still burst CPU for too long and they'd eventually tell you to upgrade.

Reply on news.ycombinator.com