Hacker News Reader

Create some sort of composite performance profile, then give me CPU information in terms of that. EC2 sorta does this[1], and although it's far from precise, at least it gives us a rough guideline and is at least an actual statement on what we're going to get.

The other issue, which I don't see Rackspace (or Slicehost or many others) addressing is the actual commit. It's fine to say "you get 2 cores", but then not tell me if those are reserved or if they might be sometimes overcommitted. This is a larger issue, because it means things might work fine... until they don't.

(I tried one provider out, and things worked swell in all our tests, but rarely in production, the entire VM would get paused for a few hundred ms ore more; something that wouldn't happen if there was a non-over-commitment guarantee. Right?)

1: "One EC2 Compute Unit provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor."

We did talk about something basically like what you describe for both slicehost and RAX, but it really doesn't solve the problem. Take that EC2 definition itself (which is frozen in time in 2007, surprise surprise), how many 2007 Opterons are there on the market now? none, they are using completely different CPUs and they've benchmarked them to be roughly equivalent, but by what benchmark? performance just isn't that simple to benchmark and there are hidden tradeoffs.

Regarding the guaranteed capacity, this is another one we spent a lot of time thinking about. At the time EC2 made the call of guaranteeing CPU share by hard-limiting the ceiling of performance by any VM. We (slicehost and RAX) took the opposite approach of weighting CPU by vm size when under contention, but allowing for burst-ability to full core power when the box was idle. That meant that in the case of low-ish utilization customers Slice/RAX VMs had a higher higher average performance but EC2 had more predictable performance.

It's a super interesting tradeoff. People would benchmark us and we'd come out way better one day but not so hot the next and that was the reason why. The flip side of that was that EC2 is stranding CPU capacity on physical hosts that are full of underutilized VMs. Even without us describing this well to the market, customers and workloads adjusted and customers and use cases migrated to each provider according to better fit (higher 'average' performance with tolerance of variability go to RAX, hardline predictable guarantees go to EC2). We had some video transcoding services that could not deal with the volatility and preferred lower performance that was predictable (and therefore kept a lot of workload on AWS).

A similar tradeoff where AWS did it one way and we did it the other is around the expectation for persistence on local disk versus ephemeral VMs. Persistence (what we did) was more like what people were used to. ephemeral is more 'cloudy'. EBS as well as hosted datastore services mitigated the difference, but the AWS approach required legacy customers and applications to re-architect for cloud and ours was more friendly to the transitional customers or people that just wanted classic feeling hosting on demand. The AWS bet was right in the sense that more of the usage of cloud (and everything actually) is in the future and not the past, and the choices we were making were more as a gap solution (but with eyes wide open about that).

It'd be cool if someone did a random survey system that launches instances on various providers, does some benchmarking, and then reports the results. It'd have to be anonymous, so that the providers don't feed it "nice" hardware.

The problem is how to make money at it.

I think that is what these guys are going for:

http://serverbear.com/

I've never heard of any of those companies. I'd be careful going with someone that you don't know a single other person who uses. Social proof matters for service businesses like hosting, because point in time performance is such a weak indicator of overall experience.

Since you're being quite candid, how did Slicehost make it in the beginning? How do any hosting companies achieve the social proof? Every hosting company has to be hosting some people otherwise they likely wouldn't exist. But maybe there's just that many customers in the world so that you haven't heard of a particular company because their customers are spread out?

Please don't take that as confrontational, I would just like to hear your opinion since you really are the authority here.

In general starting from zero you'll find some customers somehow, even just randomly and then you have to be awesome for them and build from there. The level of churn and flux at the low end of hosting is actually pretty high. You could put up little more than an unbounce page and probably get signups with credit cards at a trickle volume.

For us at Slicehost, we had an amazingly fortuitous start, due in equal parts to strategy and luck. We saw pretty clearly that rails was picking up steam really quickly and the hosting options were pretty shitty (shared hosting at the time didn't support the versions of ruby and rails people needed not to mention the memory hungriness of the framework and dedicated was still fairly pricey with 100-200/month being dirt cheap).

So we picked a really ripe initial niche market to spend time making ourselves visible in, which we did in forums, chat rooms, etc. The luck came in that we got some pretty vocal early customers who all had a great experience and evangelized us. That was lucky because either of those factors could have easily gone the other way. They could have been quiet customers or we could have had early blips in service (we had plenty of later blips, we just had a nice patch of initial smooth sailing).

But that's my point: You can't benchmark your way out of an overcommit scenario. It won't fail until it does.

You could test (or run in production) for a month, and every day things work fine. Then for whatever reason (extra sales, internal policy changing, other customers' work profiles increase) you end up CPU starved.

Without a commit/guarantee, "Past performance is not an indication of future results."

...every year processors change...300 variations of the same 'size' instance...

Shouldn't it be four variations? (Or maybe only two if you count a tick and a tock as the same.) Would it really kill providers to offer something like Nehalem-1C-4GB and SNB-1C-8GB?

for just cpu change, less than 300 but certainly more than 4. In just the 2 year from 2006-2008 before we exited to Rackspace we had more than a dozen cpu variants in production. We loaded the machines in a way where 'roughly' (with some hand-waving) things were the same for a given VM machine to machine, which allowed us to keep charging 20 bucks a month for the same instance regardless of which physical host it was on.

It's 4 CPU variations if you have purchasing agreements in place. If you're buying off the shelf from the vendors then yes it can vary much more. This is even evident in the desktop market. Compare the Dell business class to the Dell consumer class. The consumer class will change frequently, but in the business class you can buy 2 year old processors to match what you've got.

Yes you can buy 2 year old processors but then you leave 2 years of improvement/price falls on the table. Is a definable 70% performance better than a vague handwavy 100%? Depends on your use case but if predictability of performance (of a single host) is a priority you should be using a dedicated box.

My point was mostly that they sell other stuff but you have to go looking for it. The business class desktops are mostly for IT standards, so they don't have to update the image so frequently.

Making major purchases once a year and always getting the new stuff, you get 4 processors over the life of the hardware warranty (4 years, which I also believe is the timeframe for capital depreciation).

Purchasing 2x a year, you'll end up with 8 CPUs in the environment unless you opt for the 2H purchase to be the same as the 1H purchase.

If you're buying new hardware every quarter, those agreements become more important if you want to keep your environment homogenous.

If you're running thousands of servers, is it better to pay a premium on your hardware to be able to get the same machines for the entire year, or do you want to incremental improvements and environment fragmentation resulting from updating? Different models sometimes have different utilities to manage them, and now you're writing abstraction layers to handle mass updates.

I agree that predictability of performance requires dedicated hardware. I'm just going from the IT perspective.

@seats I've been a Slicehost customer for 3 years. Was very happy with the service until bought by Rackspace. Currently looking to migrate my servers to somewhere else, blah blah.

Any interest on your part in starting another VPS hosting company? I figure if you did it right once, you can do it again. You'd likely have a customer in me.

Thanks that's nice to hear.

Not in the cards for me. Technically still under non-compete, but even if that weren't the case at this point I am connected to so many other hosting providers that I'd rather just support them in doing the right things and work on other stuff. Right now I run a startup accelerator and am spending my time as a full time seed stage investor.

What's your beef with the Rackspace service right now? As far as 'big' providers, besides the obvious AWS and Rackpsace, you should also consider SoftLayer and Joyent, and depending on what you are doing the PaaS providers are all getting pretty solid now - Heroku, AppFog, DotCloud, GAE even Azure (node is a first class citizen on Azure now).

If you want something that feels really VPS-y that is run by a Slicehost-style team, Linode is a definitely a solid option (mentioning them first since this is their thread). I also am a fan of DigitalOcean (they went through our accelerator and I've spent a lot of time with them), and 6sync is run by Mario Danic who was a super active early Slicehost community member.

Thanks @seats. I was a Rackspace customer with via a previous company a few years ago and wasn't happy with their reliability and pricing. I'm leaning towards Linode because they offer the most similar Slicehost experience and have excellent pricing. I don't have personal experience with their reliability yet, but from my research its top notch.