How DreamHost Builds Its Cloud: Selecting Hard Drives | Hacker News Reader

How DreamHost Builds Its Cloud: Selecting Hard Drives

dreamhost.com

85 pointskungfudoi55 comments

There's something I don't understand about the enterprise vs consumer (or "client") SSD classification: According to the widely quoted JEDEC standard, JESD218A, [0] these are the date retention standards:

Retention Use (power off):

* Client: 30° C, 1 year

* Enterprise: 40° C, 3 months

It's not a typo, I've seen it in many places: It appears that the power-off data retention standard is only 3 months in enterprise drives (1 year in client drives isn't great either).

Can anyone explain it? My only guess is that the "enterprise" classification refers to drives designed to churn data 24/7, not store it long-term. But I think many people understand the term 'enterprise' to mean much greater durability. I'd be very disappointed if I experienced dataloss from leaving any drive on a shelf for 3 months, much less an expensive enterprise drive. (Or maybe I misunderstand what's meant by Retention Use (power off)?)

[0] Available here: https://www.jedec.org/standards-documents/results/jesd218b.0... and quoted here in the table here: http://www.kingston.com/us/ssd/enterprise/best_practices/ent...

The main difference is enterprise drives support time limited error recovery, this is a feature that allows for reliable use in conjunction with a hardware RAID controller.

It seems I misread the question, TLER is what typically differentiates an enterprise hard disk from a consumer hard disk. The primary differentiator between an enterprise SSD and consumer SSD has typically been write endurance.

Retention is related to temperature. Consumer drives may last less than 3 months at 40 C, and enterprise drives may last more than 1 year at 30 C. Both may last even longer at 20 C.

How many studies have been done verifying that enterprise SSDs perform significantly better than consumer SSDs in any metrics?

My belief is that the consumer SSDs actually have very similar performance and reliability requirements to the enterprise because they are generally deployed without redundancy, and so any failures can be catastrophic for consumers. And then there are lots of consumers that will go on Amazon or Newegg etc. and leave very bad reviews.

So I personally believe the 'enterprise' distinction is mostly a scam, and developers who are looking for inexpensive hosting have been mainly waiting for A) datacenter providers to realize that or B) a slow market perfusion that mitigates it.

> How many studies have been done verifying that enterprise SSDs perform significantly better than consumer SSDs in any metrics?

Define perform. But short answer, lots.

To be fair, consumer SSDs these days are pretty solid, and will perform great for most workloads - even many workloads which enterprise drives would be used for.

The big difference though, is with extremely heavy write workloads. The endurance of enterprise SSDs in terms of how much data you can write before the drive will fail, is a couple orders of magnitude greater than consumer drives. For example, I have an 800gb Intel S3710 SSD. It's rated to handle 10 DWPD (drive writes per day) - meaning that one could write 8,000 gigabytes of data a day, 365 days a year, for 5 years (~17 petabytes), and the drive will still be chugging away just fine.

A 1TB Samsung 850 Pro (which has some of best endurance of consumer ssds), on the other hand is only rated to 300 terabytes of data written. So the enterprise drive has a 60x improvement in endurance.

Do those ratings reflect actual tested durability or mainly marketing?

What is the real average durability requirement?

Have there not been some relatively large providers using consumer SSDs in the datacenter successfully for some time?

> Do those ratings reflect actual tested durability or mainly marketing?

They reflect what the manufacturers will offer the warranty until. A quick google search suggests about 2 petabytes of writes is the max you can get on a samsung 850 before it completely fails (and after 1 petabyte you will have many failed sectors and data corruption starts creeping in)

At the same time, I bought my S3710 used and when I got it, it had almost 8 petabytes written and it's still just like new.

> Have there not been some relatively large providers using consumer SSDs in the datacenter successfully for some time?

Yes, that wouldn't surprise me at all. Like I said, for many workloads, consumer SSDs are acceptable. But that doesn't mean enterprise drives are a scam.

I noticed that the Samsung 960s are rated at almost exactly 0.3 DWPD, which happens to be the standard endurance for "read-optimized" SSDs, even across different generations of flash. This looks more like market segmentation than a measurement.

It's rated to handle 10 DWPD (drive writes per day) - meaning that one could write 8,000 gigabytes of data a day, 365 days a year, for 5 years (~17 petabytes), and the drive will still be chugging away just fine.

It actually means something more like "we don't feel confident that the drive will last more than 5 years at 8TB/day." --- and if you do the maths, assuming perfectly even wear, this means they're rating the flash at ~20K cycles/cell, which sounds a bit ambitious for 20nm MLC considering that ~10K cycles was the norm for 65nm MLC several years ago.

1TB Samsung 850 Pro (which has some of best endurance of consumer ssds), on the other hand is only rated to 300 terabytes of data written.

That totals around 300 cycles/cell, which seems like 3-bit/cell ("TLC") flash to me, or cheap small-process MLC.

One big difference (from my experience) is that consumer SSDs would go into the market first while enterprise products would continue to test for 6-12 months for more maturity. The consumer SSDs would go through frequent updates of firmware and flash revisions while the enterprise products would be frozen for much longer. With an agreement, you can guarantee the exact same product is available many years later. When I used to do SSD firmware development, I would back-port particular fixes to an older version of firmware so a customer could minimize risks of changes. For consumer side, we would just say try out the latest version.

I think the market is going through a shift as more redundancy is used at the server level and the SSDs are considered more as a commodity.

As Kephael and the article point out, one clear advantage of enterprise drives is that they typically have capacitors that allow them to clear their caches in the event of power loss.

> How many studies have been done verifying ...

With due respect, what verifies the allegations in the comment?

While the enterprise vs consumer distictions seems to have been mostly a labeling and market segmentation exercise of the past (now we have a more differentiated landscape with NAS 24/7 etc), the story for SSDs is different.

From the technology side, there is at least the amount of extra flash that the drive contains and that it can use for better performance and longer life times (TBW rating).

Our anecdotal experience is that there is quite a noticable difference between drives: - 99.5 percentile latency during normal operations. Some enterprise drives guarantee response times, others are all over the place. Note that consumer drives often claim a much higher maximum IOPS rating whereas you can get enterprise drives with a sustained rating. - drives hanging for >60s, presumably during internal cleanup. - firmware stability, drive getting unresponsive when maxing it out over longer periods.

Reading this i really wish that Sony was not so bloody proprietary with their multi-disc BD storage format.

A consumer drive will spend its live either powered on or at "room temperature", but an enterprise drive is very likely in a data center inside a server or disk array, and much more likely to experience higher temperatures even when powered down.

Those are the same retention ability for typical flash. Every 5C of storage temperature cuts data life in half. (A higher write temperature can lessen this, but you wouldn't want to depend on it in a typical server scenario.)

But they don't want to assume all devices will behave that way. Better to specify the server scenario at realistic server temperatures.

I think "enterprise" really means "designed for living in a datacenter", with its accompanying temperature-controlled environment. If you want durability/reliability/ruggedness, "industrial" is the keyword to look for.

For hard drives, 'enterprise' has traditionally referred to durability. For example, they've had a much higher MTBF.

It's been mentioned below that differences in temperature matter a lot. But there's also the fact that erasing a flash block inherently damages it, progressively reducing its ability to retain data.

So I wonder if there's a tradeoff between longer data retention when powered off (by using higher write and erase voltages) and longer MTBF (by using lower voltages). I could be completely wrong here, but if there is such a tradeoff it would make sense for consumer and enterprise drives to make different choices.

On that note I've always wondered if putting an SSD in an oven at the highest temperature just before damaging it could revive it from being written to and erased too many times.

The same mechanism that makes SSD's not retain data (that is, the electrons escaping) and the same thing which makes them not retain the data that you do want to keep would presumably also make the electron buildup that you get that kills the cells abilities to continue their write/erase cycle.

This is only my personal experience, bu I am embarrassed to report that I have been in DreamHost customer for over 10 years for hosting some small and unimportant personal projects. I have been consistently disappointed with the level of their engineering and systems. Their systems are unreliable, their custom build ticketing system is atrocious. I would take any technical advice from them with a handful of salt.

You'd be better off with a $5/mo digital ocean VM.

Not at all. A VM is quite different to a shared host. There is more work and maintenance involved. For smaller projects its not worth it.

Take a look at Webfaction. It's dev friendly hosting, great support, has lots of 1-click installer if you need them.

+1 for webfaction. Their stuff just works and the staff is super helpful.

As a counterpoint (and this is only my personal experience), but I'm happy to report that I've been a DreamHost customer for over 10 years for hosting some small and unimportant personal projects. I've been really happy with everything. Not sure how our usage of their service differs (probably lots!), but I would hate for people to dismiss DreamHost without hearing about a positive experience with them.

Also been a customer for over 10 years. Mostly quite happy.

What bugs me about them most right now is their new control panel design. The log out link is at the bottom left, so it gets pushed off the screen when you expand the menus.

I have sent at least 3 tickets to them over the course of about 4 months telling them that they need to move the log out link to the top of the page, somewhere in the copious amounts of empty space they have up there. They have responded every time saying they would look into it, pass it along, etc. But nothing's changed. I could fix it myself in 30 seconds by copying and pasting the code in the template.

And I can't help but wonder, if they are so incompetent as to not realize that, when the log out link gets pushed off the page, their design is broken, what else are they incompetent with? Security? It doesn't take a rocket scientist to see how broken it is--do their middle managers or higher-ups not recognize the problem? If not, what other problems do they not recognize? And if my messages to them are not getting through to those who could fix them, that's a problem in their support system.

So I would keep an eye on them. I hope they are not beginning to decline.

I have been a customer for about 9 years now and I similarly host small, unimportant, and friend's projects (The "hey you know websites, I want a website" and I setup wordpress and forget about it). I am generally happy with them, their control panel is kind of shit but they are worth every penny to me. I don't have to think about them except every 2 years when they bill me. I have somewhere around 20 websites for other people on there and every time it get's close to the renewal date I consider throwing it all on DO/AWS but honestly the costs would be close and support/setup would take a lot more of my time so ~$200 is an easy sell for 2 years of peace.

I started with them in high school when I was just getting started and I don't use it for my own real projects anymore but it's perfect for friends/family and the occasional wordpress I need to throw up. One last thing, I also used bluehost for a while but I found dreamhost to have a better interface (even though they are both pretty bad)

their shared hosting product is very, very different from their DreamCompute offering. completely different everything, aside from the datacenter space.

Same company same standards?

Agreed, and they frequently make changes without warning, then send out an email, "sorry if you didn't receive notification this was happening". They make it sound like it's my fault I didn't hear them, when I'm pretty sure they just never said anything to begin with.

Azure has been more reliable, faster, more convenient, and cheaper.

No need to be embarrassed.

I've been with them for about the same amount of time. A few domains registered, personal email, a few website projects, a couple cron jobs, and an OwnCloud install ...

Generally they deliver. Not spectacular, but much better than any of the other shared hosts.

The reason why I haven't moved off them completely is because for smaller web projects it's easier to have everything in one place, ready to go and managed, without dealing with deploying and maintaining the stack (which can be overkill for hobby stuff).

All of the other shared hosts went to shit about 5 years ago. Dreamhost may eventually get there as well, but for the time being they are still decent, and get the job done.

> All of the other shared hosts went to shit about 5 years ago.

I'm not sure what your requirements are, but I've been very happy with nearlyfreespeech for a few years now. I just host a simple static site so YMMV.

My requirements are all of the shared hosts that got acquired by Endurance International Group: https://en.wikipedia.org/wiki/Endurance_International_Group

Some say EIG killed shared hosting.

I switched to A Small Orange years ago when EIG started swallowing up the world. ASO is fantastic and now I use them with all my clients and startups too.

Side note: an EIG company has given me free hosting for years because their botched takeover of my previous favorite caused them to stop billing me.

I've also had really poor experiences with Dreamhost.

I had some clients on a Small Orange. They used to be great.

EIG took over a Small Orange in 2010, but thankfully left them autonomous for while. Unfortunately in 2014 Douglas Hannah (ASO founder) left and since then things have gone downhill.

This happened around the same time EIG took over HostGator. HostGator founder Brent Oxley and (no coincidence) friend Douglas Hannah have gone on to start a new venture with their big time profits from selling out to EIG.

2014 and 2015 were riddled with major outages at both A Small Orange and HostGator.

I'm sure you remember the 4-day Small Orange VPS outage of Christmas 2015 ?! That was fun.

I got my last client off them after that and never looked back. Just now I looked at their status page and looks like they never really recovered: https://status.asmallorange.com/

Oh no how did I miss that acquisition?? Thanks for the heads up. Are there any decent independent domestic US providers left??

Yeah over the years they have had some major WTFs. Most recently one of their email clusters supporting thousands of customers went out for 9 days. http://www.webhostingtalk.com/showthread.php?t=1588201

I'd take the article with a pinch of salt, they admit they haven't got anywhere near capacity yet, and it doesn't seem like they'd done any real load testing before "final layout". They could easily go the way of most low end VPS providers, day one benchmarks look great and then a few months later when the node fills out and people start using it performances sinks with no lower bound to be seen anywhere.

Add this one, their US-East-1 cloud zone has been down for 24 hours now. My instances are in limbo where no option to just move them to another zone.

http://www.dreamhoststatus.com/2016/10/11/dreamcompute-us-ea...

> I am embarrassed to report that I have been in DreamHost customer for over 10 years for hosting some small and unimportant personal projects

No need to be embarrassed. I used DH for many many years, starting probably about 13-14 years ago. Since moved to Linode and DO in order to fully control a stack even for small personal things, however fond memories are having a managed environment for a very low price, wildcard emails, bundled domain, and mostly, sticking to their promise which got me to sign-up in the first place: any new pricing plan will be offered to existing customers at the same rate as new customers, which 10-15 years ago was almost unheard of.

Check out DreamHost Cloud if you need full control of the stack (based on OpenStack, you get more than DO and Linode at a better price)

I recently looked at SSDs for a project I'm working on. By far, the biggest performance impact is in using PCIe rather than SATA interfaces. (The performance boost also may rely on M.2 and/or NVMe; because I don't have a PCIe controller on the system I was working on, I didn't look into it further). An example:

Samsung 850 Pro, reputedly one of the fastest SATA SSD drives (SATA 3, 2.5", AHCI):

* Sequential read/write: 550/520 MB/s

* 4K random read: 100K IOPS

Samsung 950 Pro (PCIe 3.0 x4, M.2, NVMe):

* Sequential read/write: 2,500/1,500 MB/s

* 4K random read: 300K IOPS

Source: http://www.anandtech.com/show/9702/samsung-950-pro-ssd-revie...

Yeah, you get a PCIe 3.0 SSD standard with the newer MacBook Pro's and it is awesome [1, 2]. I have totally forgotten about slow disk access.

[1] http://www.apple.com/ca/macbook-pro/specs-retina/

[2] http://www.computerworld.com/article/2900330/apple-mac/holy-...

I use the 850 Pro, and it is very fast, but has many issues with Windows 7 (which is what I have).

Care to elaborate? I was about to pick one up but now you have me hesitating.

I'm unaware of negative interactions with Win7 and the 850 Pro, having used one in my Win7 desktop for over a year at this point.

I can't immediately find anything but one-off reports online that don't line up with each other, either - maybe they're thinking of the 840 line?

Sorry, that was a typo, I have a 950 pro, and Windows 7 doesn't include an m.2 driver, which causes constant problems with booting and starting up after crashes. The 850 and 840 are both great drives.

I asked about it on Digital Ocean User Voice https://digitalocean.uservoice.com/forums/136585-digitalocea... and also the Linode forum https://forum.linode.com/viewtopic.php?f=7&t=14058&hilit=NVM...

So far out of the whole internet, there was no response, except for one guys saying it was really silly, but he also said that SSDs were really silly.

So as usual it seems I am wasting my breath trying to interact with people.

The article acknowledges the availability of NVMe, but the price per unit of storage is far higher than for enterprise SATA SSDs. Cost matters, especially when you're buying thousands of units.

I've been with drramhost for at least 10 years, but Im about to move everything because of their recent move from open source webmails to only atamail or whatever the new one is called. I spent a lot of time locking down my squirrelmail install, only to login one day and be greeting by a completely different client. Sure they had warnings, but none of those warnings clearly stated the forced move, I thought I could opt out.

As for the rest of their service, Their actual server performance is decent, but their VPS offerings dont seem competitive these days, so Im strugling to find a reason to stay.

I had their VPS, but they do not give you root access. (In fact removed root access from one I used for a few years, screwing a bunch of my stuff up). Kind of a strange definition of VPS they have now.

This is managed VPS, which is kinda convenient if you don't want to manage servers yourself. For the more "normal" unmanaged VPS with root access, there's DreamCompute. It's even cheaper than DigitalOcean — $4.50 for 512 MB. I recently switched https://iwl.me to it, and it works well.

That is fine, but I was using it before those restrictions, which is annoying. Then I switched to Dream Compute and lost the email hosting for the domain.

Oh, I didn't know it was unrestricted before. (Re-read your comment) Yeah, not good of them to remove such a major feature if it was there before.

Why did you lose email for domain? It seems like I still have this option for the domain on DreamCloud (although I also still use their shared hosting).

Forgive my ignorance, but how is this "cloud" instead of VPS?

Reply on news.ycombinator.com