Seagate will bring a 16TB hard drive to market within 18 months | Hacker News Reader

Seagate will bring a 16TB hard drive to market within 18 months

techspot.com

91 pointsdoener111 comments

Random: I had a professor in college who had a bunch of industry experience. He said once that working on computer equipment was really annoying because of Moore's law. You bust your ass for a year-year and a half, make the computer literally twice as fast, and in the end the execs just sort of say "well of course you did. Moore's law said you would." Not sure if true but definitely a little funny.

The corollary to that is that in (a lot of) software your application's performance can plummet over time and nobody notices, because it appears to be holding steady. "But AcmePro Enterprise Plus is faster than it's ever been," you counter. Try running it on a 486DX4 with 32 megs of RAM like you did with version one. We'll wait.

AKA Gate's Law

* Gates' law

> * Gates' law

Actually, it's Gates's law. The "s'" construct is for plural possessives (e.g. "the three organizers' books")

In most style guides, when the name ends in "s", you still use an extra "'s" - the exception is for names in antiquity, when you don't (e.g. "Achilles' Heel").

Though this is becoming less prevalent on the Internet.

I was under the impression that both "s'" and "s's" were correct in this context with the caveat that "s'" could also be used for plural possessives.

I'm kind of hoping that SSD capacities go up or prices go down. Especially for laptops, in 2017 it feels like the baseline for the storage capacity should be 1TB. Yet here we are, struggling with 256GB, like desktops from 2007.

And don't get me started on RAM, where 8gb should be the norm even for cheap laptops...

They've dropped enough that it's cheaper for us to purchase all 1TB SSDs to replace our 650GB 15K SAS disks in our SANs, after two years our drive failure rate has dropped by nearly 900% with a 5-40x performance gain depending on what you're measuring and the drives are about half the price of what the 15k SAS drives cost, to top it off our SANs used to run so hot you could barely touch the rear or the servers and now they're dead cold 24x7.

How does something drop 900%? Did you mean it's not at 1/9 of what it used to be?

I think the poster meant -90%, so 1/10

I'm sorry and thank you, yes I did!

Yeah, but you're comparing SAS disks with server SSDs. I'm talking more about consumer grade SSDs. I feel like the price curve still starts at a quite high price point and cheap or medium laptops can't reach that point, except for very low storage capacities.

By the way, I'm talking about $800 laptops, not $1500+ :)

No I'm not, we use 'consumer grade' (which is kind of a bs marketing term if you do your homework to ensure you get a quality brand and model) ssds in our SANs, we run 960GB Sandisk Extreme Pros in our first Gen and Micron M600 or Crucial MX series in our second gen SANs, they perform great and we've had almost no failures over the past two years of hammering them. We do keep the firmware up to date which I think a lot of people are slack on. We tried some Samsung Pro SSDs in a POC and had major issues with their performance over time and several DOAs so we're lumping those in with other Samsung hardware quality and not going near them.

I wouldn't count on it. There's an industry-wide SSD shortage happening and prices are actually on the rise. This is a temporary situation while SSD manufacturers retool and ramp up 3D NAND but neglected to ramp up existing 2D NAND to meet demand in the mean time.

http://www.tomshardware.com/news/ssd-hdd-shortage-nand-marke...

I've heard that HDD prices might rise as well, but I can't find a source for it right now.

Thanks, I was in the market for an SSD for my desktop and was wondering why I haven't seen decent prices anywhere.

Scarcity not shortage

Scarcity is based on the premise that there really are a limited number of goods or services. On the contrary, a shortage is backed by the seller's decision to withhold the sale of certain products at their current price tag. Shortages can be solved by raising prices.

I've never heard of that distinction before.

What about the phrase 'there were food shortages during the war'?

In World War II, the government combined rationing with price controls — limits on what people could charge for various goods — in hopes of keeping the cost of living reasonable. This poster urged people not to undermine price controls by paying premiums for black-market goods outside the rationing system.

src: http://www.learnnc.org/lp/editions/ww2-rationing/5922

> 8gb should be the norm even for cheap laptops...

Why? For example, why now, why not two years ago, or in two years time?

I haven't paid much attention to specs recently, but looking at the Thinkpad T series which I'm typing this on (a 2010 one with a decent keyboard)

The standard for a 2017 T460s is 8G, maxed out at 20G The standard for a 2010 T410s was 2G, maxed out at 8G The standard for a 2005 T43 was 512M, maxed out at 2G The standard for a 2000 T21 was 128MB, maxed out at 512M.

So a 4 fold increase in 5 years seems a historic trend for laptop memory sizes, but that would mean 8G as standard in 2015. The maxout level has tailed off recently too - I'd expect a max out at 32G now, not 20G.

Presumably the demand appears to be dropping, that said my uptime is only 5 hours, and memory use isn't great Mem: 7779 5269 277 713 2232 1437

Browsers use tons of memory, at some point I'll have to buy the bullet and downgrade the keyboard, looks like 20G is the way to go.

You can put a 16GB chip in that 460 to max it out for $150. In the days of DDR2 it cost $800 to max out a laptop with 8GB

The non-s versions of the T-series can do 32 GB, the 20GB limit on some Lenovos is due to them only having one memory slot and 4 GB soldered in.

On the other hand, even some 2010 laptops support 32GB max, the most on current ones is as far as I know 64GB, so yes, the max curves certainly have flattened, but I think reasonably so.

I'm not sure what the price threshold is for me to actually buy all the SSD storage I need, but Dells pricing error a few weeks back was definitely below it ($20 per 256GB drive). Unfortunately my order got canceled, so I'm still rocking a 120GB ssd and a bunch of shitty slow spinny disks.

You can get a 4TB SSD. They are a little pricy, but they do exist.

Too bad the $/TB isn't dropping as much as before. $30-40 / TB seems to be the floor...

http://www.theregister.co.uk/2014/11/10/kryders_law_of_ever_...

I paid $1000 for my first HD. It was 40 MB.

About 15 years ago, the company I worked for paid $1,000,000 for a 1TB disk array. Today I have several times that capacity laying around my apartment; I haven't even bothered to hook some of it up.

As someone who installed a lot of NetApp filers back in the day that had 1T of storage each I would also point out that it took up the whole rack :) Yeah, back when the move from 9G to 18G drives in the array was a big deal...

Same, I have 11 TB from boxing day sales sitting on a shelf.

My lab paid 35,000 for PDP 10mB disk in 1976. It was the size of a washing machine. But it did allow us to increase our research.

I had one of a similar capacity. It weighed as much as a brick (perhaps more).

https://en.wikipedia.org/wiki/Hard_disk_drive#/media/File:5....

I paid something like $360 for a full-height 5.25" 30MB Seagate for my first HD (to add to my dual-floppy Zenith AT).

These helium-filled drives always make me twitchy and worried. I know they've come up with new seals to keep the second-smallest atom from leaking out, but I'd really like to know what the lifespan of these drives will really be.

That said, in the datacenter environments where they're pushing these hopefully the drives will be better monitored and replaced regularly as opposed to the consumer and small business market where there are still people who will give up their XP when you pry it from their cold virus-infested 15-year old HDs.

Fun fact: the helium atom is actually much smaller than the hydrogen atom. See https://www.webelements.com/helium/atom_sizes.html

Not clear where that's coming from, that site lists the bond length for HeHe as 300pm but for HH it's around 75 which would make Hydrogen ~1/4 the size of helium.

On the scale of individual atoms I don't remember enough about my chemistry and physics courses to recall whether there's supposed to be much physical difference between the "size" of the region where the first 2 electrons are between different atoms, nor do I know if that's being conceptualized as orbits, shells, abstract concepts of energy levels of something even weirder these days.

I think the helium in the harddrive consists mostly of single independently moving atoms, not molecules of two. Those have very small binding energy, much smaller compared to kT.

It's not an original thought, but for my home / office environments (ie. non-data center) where most machines have at least one 5.25" bay, I'd be really happy with a larger, slower, higher density, disk.

I appreciate most vendors are selling to the commercial market which eschews this old format, but it's still hard to buy a tower case that doesn't include at least one 5.25 inch bay that never gets populated.

(Nota bene - I'm referring to half-height 5.25". I've long since dumped my impressive stash of 2GB full-height 5.25" SCSI disks.)

Biggest problem with large spindle is that noise and heat dissipation (both mostly due to air friction) are roughly proportional to the square of linear speed of the outer part of the platter, and bigger the platter, more linear speed you'll get at the same angular speed. On the other hand, you want to crank up angular speed to get any decent random IO operation/second value.

Last 5.25" HDD I've seen was Quantum Bigfoot CY [1], (which happened to be half-height FWIW), and this thing was pretty noisy even at 3,600 RPM. Dividing this by 60 (seconds in one minute) gives you theoretical maximum of only 60 random IOPS. Not exactly stellar, and as far as I remember, that was substantially lower in practice.

[1] https://en.wikipedia.org/wiki/Quantum_Bigfoot

The Bigfoot failed in the market because they crashed and ate your data. The time might be right to try again with better execution.

The current use for using spinning disk is for capacity applications like archive/media storage, so the speed tradeoff would be fine.

I can imagine a 5.25" drive populated with a couple(3-5) sets of ~1.8" platters, you could also have internal raid speeding things up and protecting from single set failures.

Why would you ever do that vs having separate disks you RAID if desired? That sounds like you'll inevitably end up with at least one failed disk in your 5.25" array, and then what do you do? If you can open it to replace the part, what is the functional benefit over separate disks to begin with?

They tried that. Quantum made the quarter-height 5.25" "Bigfoot" drives for a little while. They didn't go far. I think the problem is that a larger disc, though it has greater area, also comes with worse mechanical stresses, plus the head has to be larger, which again means greater mechanical stresses and distortion under movement, and can't seek as quickly. There's a reason drives have gotten steadily smaller; otherwise they'd be making big, flat 8" or 12" hard drives for really high-capacity needs.

I bought a cheap adapter bracket and put an ordinary hard drive in mine. Not the most efficient use of space but it means it isn't completely wasted.

Those bays are for DVD drives, not hard drives.

Do you mean it's against the natural order of things, or that there are technical obstacles like limited airflow (cooling)? A good temperature monitoring and throttling system could be a reasonable failsafe for the heat.

BTW does anyone know why current rotating HDD's are always fixed RPM?

I was responding to this:

> it's still hard to buy a tower case that doesn't include at least one 5.25 inch bay that never gets populated

Do many people have two optical drives? I have two exposed 5.25" bays but only one DVD reader. I would happily put a slower 40TB hard drive in the other if the price were right.

> Do many people have two optical drives?

I have no idea. But Ron's third law applies: it is usually better to have it and not need it than to need it and not have it.

Which quite a few people don't use anymore, and thus leave these slots empty. (I didn't plug mine in after the last motherboard change. Didn't have a fitting cable at hand, and never needed it since, so no pressure to fix that)

Yes, I know that. The point is if you want to use a DVD drive then you must have a 5-inch bay. You can't shrink a DVD drive the way you can shrink a hard drive because you are constrained by the size of the media. So the 5-inch bays are there simply to leave open the option of installing a DVD, not so you can fill them with hard drives. You actually can put a 3-inch drive in a 5-inch bay using readily available adapters if you really want to. But asking manufacturers to make 5-inch drives so you can fill 5-inch bays is silly.

Will this be SMR? That seems like an important variable.

When will prices of SSD catch up to rotational disks?

caveat: I'm in the storage/file system industry.

Everyone in the industry is looking at this. There is a great graph out there by one drive manufacturer that overlays $/GB for different HDDs and SSD technologies. For example, 15K and 10K RPM drives no longer are cost effective to produce. You are better going with SSDs. 7.2K will be next and some of the "lower end SSDs" are encroaching on that. 5.4K will likely live a longer time and if you keep "HDD as archive" you might project these drives to be alive in the foreseeable future (then factor in SMR, HAMR, then SMR plus HAMR.... and $/GB gets pretty cheap). After all, if you aren't keeping 100% of your data "in flight" keeping it at rest on a cheaper medium seems a sane thing. (Tape backup systems are _still_ for sale and in use...)

Another interesting back of the envelope to do is look at Fab capacity for SSDs and compare it against HDD capacity sold. How much Fab capacity has to be available to serve the total storage demand? If there is a shortage of Fab plants, then a price premium for SSDs is easier to maintain (and may benefit some flash manufacturers in the short term to keep the price premium in place). Looking at the economics of building enough Fab capacity gets interesting quick.

> (Tape backup systems are _still_ for sale and in use...)

LTO seems to tick a lot of boxes in the archive business. I remember people trying to sell a MAID (massive array of idle disks) some years ago -- something like 100 disks in 4U of space. They came of blades that each held about 8 disks, and there were about 15 of them in a 4U row. With a controller in the middle it meant you could get somewhere in the region of 1000 disks in a rack.

The power consumption would be obscene, but that was fine, as the disks were kept idle, in fact the system could only power about 25% of the disks at once. The idea was you don't need access to the majority of files, and a 10 second latency when you do isn't going to hurt (far faster than tape)

At the same time tape manufacturers were pushing LTO-5 and LTFS, arguing that was the better solution for archive, and after a misstep with LTO6 (launching at 2.5T rather than 3T), LTO7 recovered.

MAID is rumoured to be how AWW Glacier works.

I've actually been wondering why is not possible to make lower-end desktop SSD drives of capacity 1TB (or more) that are "better" then spinning drives.

I have a samsung 950 pro NVMe drive in my development workstation and I can't statute it without a synthetic workload. There are times I need that for work, most times it's over kill.

For example 1TB SSD drive that can do like 300MB/s sustained transfer with 3000 to 5000 iops. So a speed that's magnitude or magnitude and a half less then current consumer SSD drives. But still a magnitude faster then spinning disk. I'm confident that would be more then enough to make most desktop environment work feel just as snappy as a SSD or within an acceptable factional difference.

If you could make this at $100 to $120 price point drives. That's like 1/2 the cost of a low end desktop 1TB SSD drive (like the Toshiba/OCZ) You'd have a hot seller in peoples gaming rigs, mid-range laptops etc.

Obviously I'm missing something here. Why is not happening? Does producing lower end flash (slower) not saving you any money?

Because 128GB piece of second/third grade flash is ~$20, and those go into cheaper SD cards/usb drives. $160 in cheap memory alone.

There are manufacturers playing aggressive race to the bottom, they make drives with neither ram nor even SLC buffer. Mostly JMicron and Silicon Motion controllers, but more and more companies jump on this opportunity like Marvell. Im not even talking about garbage bin chinese "vendors" like kingspec/kingdian/kingrich/ki..you get the picture, I mean SanDisk Z400 or OCZ TL100, proper manufacturers.

At the end of the day those are pretty much USB pendrives with switched PHY, you can expect same prices and performance. That also means drastically lower write endurance, ~20-40TB Total lifespan. ~20GB/day in an age of Windows 10 writing ~1-2GB of logs(you know, so they can upload them to ze cloud, harmless telemetry ..) alone every single day.

Thanks, that was helpful.

Such drives technically existed in the early days of consumer SSDs (2009-2010), but even then, the limitations in speed were due to poor controllers and firmware (certain Jmicron controllers, if anyone remembers).

The slow flash chips that you're talking about probably don't exist in sufficient numbers to warrant making a separate product from them. Let's say their lithography produces 5% of slow flash chips. It is probably more cost efficient to sprinkle those chips in existing product lines as spare capacity (all SSDs are overprovisioned, afaik), rather than making a whole separate product line out of them - a product line that will, by definition, be low end and thus have tiny margins.

> be low end and thus have tiny margins.

But high volume, it could be in every laptop under $800 price point. A lot of these guys already make low margin high volume products (like Samsung and TVs) and that's okay.

There's no 'but' here. The fact that it's not happening is pretty good evidence that economics of those drives don't work out.

I also explained why it can't be high volume - there might not be enough actual low speed chips to make high capacity drives in volume.

I'm not sure if you understand the economics, but no sane for-profit company will trade margins for high volume.

Sigh. There's no reason to be be a condescending prick (ad hominem).

Simply asking somebody who works in the industry why it's impractical to segment the market by making slower chips. Maybe it's not possible or they cost the same, but I'm just speculating. Hence wanting to find out and understand.

> I'm not sure if you understand the economics, but no sane for-profit company will trade margins for high volume.

Business don't always get a choice, or do make that choice (based on game theory). In the case of SSD vendors it's a decently competitive market with many vendors (Samsung, Intel, Toshiba, Sandisk, ...). In fact it looks like Samsung is going this route: https://www.extremetech.com/computing/236260-samsung-plants-... . Which they might as well do because they one of the largest producers so it makes sense to put pressure on your competitors if (if you can sustain it).

~2019

Will they?

I just found out today that Seagate bought Dot Hill.

Wow HDD's really are the new tape drives

it's a shame that HDDs fail/lose integrity so much more quickly than tape cartridges used in an equivalent manner :(

but if you put them on a shelf, then the life is roughly the same :D

> but if you put them on a shelf, then the life is roughly the same :D

Careful, that's not always the case.

Anecdotally, I lost a few TB's of data recently after assuming my drives sitting on the shelf (inside anti-static bags) would be fine after only a couple of years. Plugging each drive in prompted the Windows popup asking if I wanted to initialize the drives! Total data loss on all 4 drives after only a couple of years on the shelf. It wasn't a big loss to me, because I couldn't even remember what data was on the drives... but I wouldn't rely on this for business data.

Tapes still have their purpose.

Are you sure it was a total loss? There are some pretty good open source programs you can run and try to recover the data as long as the drive still spins. I recently went through some of my own drives (old backups from many years ago as well as one 2 TB drive in my NAS that decided it wanted to start failing). Photorec[0] and ddrescue[1] came in handy. They can take a long time to run (I did multiple ddrescue passes on my 2TB drive and recovered all but 16 MB, but it took about a week's worth of time), but it may be worth a shot.

[0] http://www.cgsecurity.org/wiki/PhotoRec [1] https://www.gnu.org/software/ddrescue/

> Are you sure it was a total loss? There are some pretty good open source programs you can run and try to recover the data as long as the drive still spin

Data recovery will likely work, since they mechanically work and Windows does see them as drives, just no data (says uninitialized).

My initial thought was it's just the partition table that was corrupted, so something like TestDisk might easily restore the table and allow me to extract my data. Otherwise something more serious would be needed... at this point though, I can't remember what's on the drives, so my effort level is minimal.

My "total loss" statement was more along the lines of using hard disks for backups. Having a corrupted partition table from the drive sitting on a shelf isn't a good thing to rely on for backups. If it were business data, it would have been on tapes instead of disks.

Fair enough. The drives I looked at didn't have any particularly pertinent information on them either, but I actually made a pretty fun weekend out of learning how to recover data and re-live some of my personal history starting from around 1999. I felt like some sort of digital archeologist -- reading e-mails and looking at pictures.

Interesting story, was it a hardware failure that caused the loss, or was it Windows deciding to wipe them? Apologies if I'm missing something obvious

disks are also much more succeptible to vibrations than tape. the only thing is that maintaining tape expertise to pull data back into modern systems is a bit difficult

> was it a hardware failure that caused the loss, or was it Windows deciding to wipe them?

No, mechanically they all work fine... just no data on them anymore. I believe the partition table became corrupted, so Windows just thinks its a wiped/new drive and asks if you want to initialize it.

In my case, I said "No", and will attempt data recovery sometime when I'm bored (TestDisk[1] might just be able to restore the partition table, allowing me to extract data from the drives then perform a real wipe, otherwise a more serious data recovery tool like StellarPhoenix, GetDataBackNFTS or similar will be required).

> disks are also much more succeptible to vibrations than tape

They sure are. There are other factors too, such as radiation (from solar flares and more), temperature, and more that can cause "bit rot", even for a drive not in use.

> the only thing is that maintaining tape expertise to pull data back into modern systems is a bit difficult

Usually not so much. Whatever utility you used to create the backup tape almost assuredly has the ability to restore data from the tape. Some even are full disk images written to tape, which allows you to put a tape drive into a new system and restore the entire thing... while others only backup data to tape, etc.

[1] http://www.cgsecurity.org/wiki/TestDisk

I wonder if this will have such high failure rates as other high-capacity hard drives. It's less cost effective to buy many smaller HDDs, but it's probably better for long-term data retention.

I wish there were a cheap and easy way to buy magnetic storage and the associated drive. I have no idea where to start, as I just want to back things up on tape.

It's less cost effective to buy many smaller HDDs, but it's probably better for long-term data retention

How so? Is the MTBF for a size 2*x drive more than twice as high as that of a size x drive?

I don't think we should really be looking at drive MTBF, but rather some measure of total expected loss. The bigger your drive, the more catastrophic the loss in the event of a failure, so even if the MTBF of a 2x capacity drive is only slightly higher, your total risk is more than 2x as high.

Only if you don't have backups. If you do have backups, the cost of loss is proportional to your I/O rate for time spent in recovery.

Hence the suggestion of RAID elsewhere in this thread, which means you might as well get more smaller drives in an array since the $/TB seems not to be dropping on these bigger drives.

RAID is a really bad idea with larger disks, as

1)not all sources of failure are uncorrelated

2) with huge disks, the rebuild time to replace that failed disk gets really really long (I would guess 2-3 days to fully write to one of these disks. Edit: rebuilding a RAID rather than writing straight will take several times longer.)

3) during rebuild all the disks are taxed quite heavily, especially if they are still performing their typical day to day IO.

The larger the disk, the more of them you want in a non-RAID unified erasure coded pool in order to manage these failures more gracefully.

I just rebuilt a raidz2 (4x4TB disks) on my home "server". Took a couple of days. I've never had even a single disk fail during rebuilding to date, so I'm not super worried about the risk of having 2x failures during the 4 days or so it takes to replace a disk (next-day delivery on the replacement and then doing the rebuild). It's a lot more "just works" than backups would be - in principle I'm not worried about high availability as such, but I am worried about faff when replacing.

Make sure you're snapshotting with exponential-style backoff (e.g. keep 24 hours of hourly snapshots, 7 days of daily snapshots, a few weeks of weekly snapshots, etc.) to guard against clobbering or deleting data.

ZFS isn't an absolute panacea. It can get very confused if you have a systematic problem one day - e.g. if you ever have intermittent bus problems (e.g. dodgy backplane), it can start to believe that that blocks are permanently damaged and it's hard to convince it to read through to the disk storage. I had to copy the superblock from an old snapshot[1] once because it was convinced all of its copies were bad. This wasn't during writes; it was during a zfs scrub.

If you don't do it regularly, act like a zfs scrub could potentially wipe out all your data. This is more likely on home hardware, without ECC RAM and less likely to be built to handle loads of concurrent I/O throughput.

Keep on using backups.

[1] Thankfully, because of my snapshotting policy, my old snapshot wasn't actually very old.

4TB with raid-z2 is perfectly acceptable, as it has lots of extra bits for the amount of down time. Also, it's ZFS, not straight RAID, which has advantages for bit flips. I still have many many 4TB drives in large RAID6 arrays. Personally, I'd stop at 4TB drives for this though.

With 16TB disks, that rebuild time goes to 4x as long; and though one could probably write 16TB in 2-3 days in one go, a RAID rebuild of more than a week sounds typical.

If you didn't have ZFS, I would also caution anyone from using straight non-ZFS RAID as a backup substitute, because there's no checkpoints and no way to stop data deletion from affecting you.

> If you didn't have ZFS, I would also caution anyone from using straight non-ZFS RAID as a backup substitute, because there's no checkpoints and no way to stop data deletion from affecting you.

The type of array redundancy (ZFS, mdadm, megaraid, etc.) does not matter. It is never a substitute for backups. End of discussion. These are two entirely different technologies meant to protect against entirely different problems.

This is a common meme I've seen constantly through my entire career, and one of those common misconceptions you have to educate customers on quite often. RAID is not for data security. RAID is for availability.

Snapshots, checksumming, etc. on ZFS are wonderful - I use it extensively. I also zfs send said snapshots to an off-site remote server, and if I didn't have that capability I'd rsync to an external hard disk or whatnot.

Even if you could somehow guarantee ZFS or your hardware would never combine to lose your data (you can't) - it still doesn't protect from your house burning down. A substantial number of people completely neglect that failure mode until it's too late.

(I'm biased here)

> I also zfs send said snapshots to an off-site remote server, and if I didn't have that capability I'd rsync to an external hard disk or whatnot.

I wouldn't consider a backup system any more that isn't able to establish full integrity of backups. It's also a kinda good idea to diversify the software; ZFS has a really good track record, but if I where using btrfs, for example, I'd make sure my backup software is not the same as my FS.

Ah, a fellow paranoid sysadmin who's been around a while :)

I do this for my "very important" stuff - e.g. documents and personal photos. That gets copied every year to a third off-site physical drive for safekeeping, and that is an entirely different filesystem. A lot of that stuff is also on google drive for the latest, so I'm not super concerned about the backup frequency there.

The standard zfs send backs up about 25T of stuff I'd certainly not wish to lose, but could eventually mostly be re-created. For my risk level zfs send seemed to be the correct risk:ease of use tradeoff for me. Plus the nice diffing makes syncing a home NAS on a consumer broadband connection not entirely impossible - where even rsync can be difficult for some of my use-cases.

Full integrity of backups is interesting - in a professional environment that means to me full on DR testing (e.g. restore your backups regularly to test them) among other things. For personal use I think automated regular zfs scrubs are about the best you can easily get.

I wouldn't consider a backup system any more that isn't able to establish full integrity of backups.

This isn't very hard to do at all. It's very easy to DIY full integrity. Especially with ZFS, where you can take a read-only snapshot and not have the FS changing underneath you while you're traversing it. Here's a previous post of my on the subject: https://news.ycombinator.com/item?id=13129294

Something as simple as the following, done independently for source and destination, and results compared, should suffice:

   find -x \
        . \
        -type f -print0 \
        | xargs -0 -n 100 -x md5 \
        | sort > /tmp/manifest.source

It's a question of costs and benefits. RAID absolutely does reduce the risk of data loss. The extent to which you want to take further steps is a question of how valuable your data is and how much of it there is. It can be perfectly legitimate to decide that, given a particular risk profile and a finite budget, you can't afford both RAID and backups and the former is better value than the latter.

If I guess right in what you mean here, then I'd say that in these instances backups are better value than RAID.

Agreed, it's never a full substitute for backups.

It is a substitute for single drive failure.

And ZFS is a substitute for some amount of data deletion (but not much).

But second location, cold copy are absolutely necessary for a backup.

RAID 6 copes with this reasonably well.

https://en.wikipedia.org/wiki/Standard_RAID_levels#RAID_6

I don't think this works. With more drives, you expect more drives to die in the same time period, so total expected loss stays exactly constant.

Yes, but they don't all die at the same time, so you can utilize a RAID or similar array to mitigate the failures at a small cost in total storage space. Ex. my 5-drive RAID array runs 5x 3TB drives and provides about 12TB of usable space. If any single drive fails, it's a quick swap with a cheap replacement. If I instead had a single 12TB drive fail, I'd be screwed.

Even if I didn't have a RAID array, any single drive failure would be a maximum of 3TB lost, vs. losing everything with a single drive. Basically it's much easier to minimize your data loss risk profile if you spread your data across multiple drives instead of relying on a single drive.

Of course, if you need 100TB+ of storage, you might as well put these new 16TB drives in an array and save on the controller costs.

And then if there's an unrecoverable read error on one of those other massive drives during the rebuild, you're in deep trouble.. and the bigger the drive the more likely this is to happen (reaching near-certainty once the drive is large enough):

http://www.zdnet.com/article/why-raid-5-stops-working-in-200...

Which is why the 3 and 4TB drives I have in my FreeNAS box at home are configured as mirrored pairs, the time it takes to resliver a drive in this configuration is already fairly long - RAID-Z or the hardware equivalent of RAID-5 would take upwards of days to complete and that's a disaster waiting to happen.

At this size, the rebuild time is a day, or even 2-3 days if it's actively being used during the rebuild. It still makes sense to do at least a 2x raid1, which rebuilds faster than a cluster over wired ethernet. But if you have this much online storage, you probably have a high availability need also, which implies cluster in case a storage brick loses a power supply or logic board. It's not just a drive that can fail.

RAID?

So RAID two big drives? What's the point?

We are at the point where drives are large enough, that rebuilding an array of large drives have a dangerously high probability that it will cause another failure in another drive, killing the array.

In the case of RAID-1, redundancy? The R in RAID?

Yes but that doesn't resolve the small vs big drive issue. You can RAID them both. The question wasn't what the point was of RAID, the question was what the point was of smaller drives.

RAID what? You trade redundancy for less space.

When you are looking at the potential of losing TB's of data it's money well spent. I ponied up some extra money to set my FreeNAS box at home up with a pool of mirrored devices because I care about my data - if you don't then feel free to save the money. (One of these days I'm going to pony up for a LTO drive to do a monthly backup and drop it in a safe deposit box, one ~$115 LTO-7 tape is enough to backup my entire storage array).

> One of these days I'm going to pony up for a LTO drive to do a monthly backup and drop it in a safe deposit box, one ~$115 LTO-7 tape is enough to backup my entire storage array

the cost of the LTO-7 drive would buy you an eSATA dock and a decent pile of 8TB SMR drives, which you could just write the zfs dump to linearly.

Data stored on tapes is a lot more stable than data on spinning disks, even when sitting on a shelf.

I certainly would not replace a tape backup with spinning disk backups - rather, I would use the disks for routine backups and tapes for quarterly archives or similar. Although, for some of our systems, we use rotated tapes nightly.

There's a good reason so many folks still use tapes for important systems.

RAID and smaller disks doesn't somehow change the dynamic of $/TB.

It does, unless you're only factoring in the cost of the drives themselves and not:

1. The power to run them, and the additional cooling and acoustic concerns as a result

2. The cost of extra storage enclosures to support them

3. The additional space used by the smaller drives (do you want 2U of 12x8TB drives, or 4U of 24x4TB drives).

If your concern is space, use RAID5 or RAID6, which offer single or dual disk recovery. There's been some concern raised, particularly about errors on RAID5 recovery - the large disk sizes may make a failure during recovery more likely, but in my experience drives more often than not fail completely and totally, all at once or close to it.

If you're extra worried about recovery though, my friend has a good setup where he keeps 1 TB RAID1 across all of his 4x8TB drives for personal files and uses the rest in RAID5 for media storage.

Yeah, that's my concern. I want reliability, super reliable. So reliable that I can live life recklessly without a backup. Backup does get expensive, managing, paying, the bandwidth, the restoring.

Increased storage reliability doesn't protect you against data corruption due to software defects or accidental deletion. You still need backups to protect against those problems.

That's the dream, but until you can guarantee the building your drive is in will never burn down / be struck by an earthquake / be broken into and cleaned out, you will need some sort of backups.

shred /array0/important && rm -rf /array0 && zfs list -t snapshot -o name | xargs -n 1 zfs destroy -r

(Or something along those lines)

Reply on news.ycombinator.com