Hacker News Reader

Comment by krob | Hacker News Reader

HGST look like the best, but they don't have the quantities of the Seagate, makes me wonder if these numbers are skewed :/

View on HN

My experience, over decades, but now coming up on half that ago, was that IBM/Hitachi/HGST are definitely worth it. I used to run a small hosting business, which had hundreds of discs, and consulting business which had clients with hundreds more discs.

Seagates, in general, could always be expected to fail in a 3 year span. Often multiple times. IBM/Hitachi/HGST, especially the UltraStar enterprise line, would basically never fail. Over 20 years and hundreds of discs concurrently running (switching out as opportunity arose), we probably RMAed single digits of HGST drives.

In comparison, while I had a few pockets of Seagate drives that didn't fail (I had 6 in a storage server at home that never gave me problems), we could generally expect 5% of Seagate drives that we had running to fail in a given year. Enterprise or regular didn't really seem to matter.

For us, replacing a drive was fairly expensive, using a limited resource (our time). So we gravitated to HGST almost exclusively.

But: We also did an extensive burn-in process before a drive went into production. Basically: "badblocks -svw" for a week. He noticed that we had some drives that would fall out of RAID arrays, but if we ran badblocks on would never report an error. My theory was that there were some marginal sectors that would bit-rot. Running a week of badblocks would exercise those and allow badblock remapping to remove them from use.

Remember the IBM "Deathstar" 75GXP? We even had good luck with those. I had one of them start freaking out, and I was aware of the "Deathstar" name, so I went to replace it with another drive I had on hand. When I pulled it out to replace it I realized it was HOT. Not in the grand scheme of things, but definitely hot to the touch. I looked up the temp specs and it was clearly above that. The box it was in had 2 5-inch bays that didn't have the covers on them. I covered those bays, turned the machine back on after re-installing the Deathstar, and the drive continued operating for another 3-5 years with no problem. Made me wonder if improper cooling was the source of those reports.

> improper cooling

You've had far more experience with this stuff than I have. I wonder what your view is on all these NAS RAID boxes and HDD longevity and heat.

I personally never saw so many HDD failures until I started running them inside a small NAS I bought some years ago. I nicknamed it my "drive killer" because I was replacing the things so often. Other HDDs I have in other (non-NAS) machines sure seemed to last much longer.

As for the Deathstar story: I had a similar experience with a Seagate. Inside the NAS, one of the SMART error counts started to rise in a foreboding way, so I replaced it (with an HGST) before it had a chance to die. Then, out of curiosity, I tried the old Seagate in a different non-critical setting (non-NAS) and it's continued to work for two more years. IDK. Too much heat?

AFAIK Google have said they never really find any correlation between SMART errors and actual drive failures.

Interesting. Have not heard that before. My anecdotal experience in that NAS enclosure with Seagate drives was that when, IIRC, "Reallocated Sector Count" started to go up week after week, the drive would fail within a few months.

It's not impossible to have a drive appear unreliable in one machine and reliable in another (if you physically move it). There could be something electrically or mechanically wrong with the machine which causes a particular drive to malfunction more frequently.

If you see this repeatedly with one machine, I would definitely consider that it is the machine which is bad rather than the drives. A power supply may be insufficient to support the peak requirements of the drives, for example. A poor mounting structure might resonate or transfer vibration or shock into one or more drive slots from the environment.

thanks for that interesting note. i had been thinking only about heat as a cause of trouble in that enclosure (an old Netgear ReadyNas).

Vibrations can be problematic in multi drive chassis. The more NAS rated drives are apparently rated specifically to deal better with vibrations and of course the NAS itself some isolate drives far better than others from each other.

It might seem minor, but just watch this rather old now but infamous video of Brendan Gregg shouting at HDDs causing significant drops in throughput: https://www.youtube.com/watch?v=tDacjrSCeq4

fun video! it doesn't take much vibration. (or he's just an incredibly loud screamer.)

As others have mentioned, vibration can be a problem in these environments. But heat is also something you want to deal with. Clean power is also something to be concerned with.

In the past I've used a lot of those Supermicro 5x3" drive carriers and they have really good airflow. This was for storage in servers I built for home use. I wrote up something in 2008 about one I built here: www.tummy.com/articles/ultimatestorage2008/

Faster spinning disks generate more heat and vibrations than slower spinning ones. 15K drives are notorious for needing good environmentals.

My current storage needs are pretty limited, which probably would make SSD attractive. Or we are getting there at least. My storage server right now has 6TB used, and around 1TB of that is backups. If I could get by with 4x 2TB SSDs, I might consider it, just for longevity sake. I'm using ZFS BTW.

Yes. I used to see equipment -- hard drives, video cards, motherboards -- burn out at least once a year, and then I installed a power conditioner on my hardware. That was 4 years ago. Last year, for the first time in 17 years, I upgraded my motherboard by choice rather than replacing it after a failure.

I built up a homebrew NAS with three of those Supermicro drive bays, and, other than the need to remove/replace screws (and not lose them for empty bays), I've really liked them.

However, I can't for the life of me find out whether or not those bays will work with 6 Gb/s SATA. I've been considering putting my Ryzen 7 board into the NAS case to consolidate two systems into one, but I'm not going to go to that effort if I have to get new hotswap bays.

The Deathstar 75GXP issues were [quote Wikipedia] "due to the magnetic coating soon beginning to loosen and sprinkle off from the platters, creating dust in the hard disk array and leading to crashes over large areas of the platters".

Perhaps heat could have accelerated the process by loosening the material somehow, but it was definitely a design/manufacturing flaw.

Anyone have a good source of HGST drives? On Amazon, it used to be you could only get them from resellers and some comments complained about getting used drives. I basically stopped buying HDs from Amazon from resellers.

EDIT:

Example:

THEY ONLY WAY YOU CAN VERIFY THAT YOU DID INDEED RECEIVE THE FULL 5 YEAR HGST WARRANTY IS YOU MUST CALL HGST TECHNICAL SERVICE, GIVE THEM THE SERIAL NUMBER AND ASK IF THE DRIVE IS A NON-OEM, OEM OR REMANUFACTURED.

I have ordered expecting the full 5 year warranty, and verifying with technical service, have discovered - one with over 18,000 hours (from Crystaldiskinfo), several OEM's (no manufacturer warranty), a remanufactured and a full 5 years manufacturer warranty. AND once in awhile I get lucky and received as advertised - a full 5 year manufacturer's warranty.

Agreed. I only buy storage for personal use, but I've learned to avoid Amazon entirely. There are just way too many reports of fake or repackaged HDDs, SSDs, SD cards etc.

I can generally get the same or relatively comparable prices at local computer stores that I trust significantly more than Amazon.

Be careful as there are reports of the same issue at Bestbuy(And will be with any store). With the 8TB easystore (and now the 10tb as well) being so cheap, there've been several reports of shucking the drives only to find a lower tier drive or just sand or whatnot.

This includes drives that were shrink-wrapped! I've heard bestbuy re-wraps returns and doesn't check them very well.

They're not going to be skewed. HGST disks are usually more expensive, so that limits the quantity that they buy. The seagate disks are usually cheaper but have a slightly higher (except when it's a brand new line) failure rate. When filling out a single server I go with the HGST disks because the premium price and quality means fewer failures, but it's more cost effective to go for lots of seagate disks when you have more redundancy and can eat more failures.

I built a server a few years ago, and I determined it was more cost effective for a drive to fail than to use HGST. It would be more inconvenient, but having a drive fail on a home server with only 6 drives didn't seem very likely anyway.

Particularly if it fails within the warranty period.

If you have a 2%/year failure rate per drive, then that leaves you with a nearly 12%/year chance that at least 1 of 6 drives will fail each year. Or a 31% chance that at least 1 of 6 drives will fail within 3 years. Or a 52% chance that at least 1 of 6 drives will fail within 6 years.

The question then is, would it be cheaper to replace that one drive or get the more expensive disks with lower chances of failing?

Isn't there a cost to time and convenience too? Buying a new disk takes time, as does rebuilding the RAID. And during that time you are vulnerable to another drive failure which could be disastrous if you only have one drive redundancy, especially during the very intensive rebuilding process.

There's also the cost of the time it takes you to replace the drive, get it replaced under warranty, etc. that takes a part of the cost of everything too. I decided that the (at the time) marginal cost of $20/disk was worth my time in likely not having to deal with it.

This makes me curious of where the cost/benefit point is between these brands of drives. Do they buy enough that the reduced cost for higher failure rate outweighs the expense of buying more expensive drives?

It's only partially a question of quantity.

The larger factor is the amount of labor involved in dealing with a bad drive.

Their (large) operation seems automated so I assume if they have a single drive failure in machine #1234 (which has 50+ drives) they have an automated way of switching off said drive. Then they leave it there until the entire rack is replaced many years later.

Other operations have to send a human to manually replace every single failed drive as soon as it fails, which is very expensive as a % of the cost of the bare drive.

They buy enough drives that there's another point of data that's needed for them. Availability of the disks. If they need to expand a few 10s of petabytes they may only be able to get enough disks from one group regardless of reliability concerns.

Thats what surprised me when I learned that they were "shucking" drives from off the shelf external drives at some point.

I agree (about Seagate being more cost effective for e.g. a datacenter). As private user I had in general bad experiences with Seagate (e.g. years ago I bought 8 2TB drives for 2 raid5 and 2 didn't even spin up, 1 failed after ~1 year) => I'm using since then only HGST and (pls. no jinx) so far only 1 has failed, but it "warned me in advance" with weird noises and terrible performance for ~1 week or so.

I now have at hetzner.de a host that has 3 4TB Seagate drives and I ended up building a 3-disks-raid1 instead of a raid5 because I'm too scared that when one drive fails, an additional one will fail as well during the rebuild-process when the remaining ones are stressed (a raid5 would then implode in this situation).

They show the rates over years, too.. Quantities may be a factor in lower numbers, but over time they are reliable as well.

Reply on news.ycombinator.com