Hacker News Reader

Comment by fnord123 | Hacker News Reader

>Might as well ask here. My 2TB Plex hard drive is getting full. What's the best solution if I want say 10TB to store my media for Plex but very hands off?

You probably need a NAS. To store 10TB securely you are looking at a bill of materials of about ~1200usd. This includes a NAS from synology and 3 6TD Western Digital Red drives. This will give you 12TB of storage space running in a RAID5 configuration.

Synology DS416 NAS diskless: 365USD WD Red 6TB: 253USD (x3) Total: 1124

For a little more you could pick up a HP Microserver for just under 400USD and run ZFS on a pool with SMBD and get about the same results but with more hands on fun.

Alternatively, just aim your requirements a bit lower (e.g. only 6TB) and pick up a 2 disk nas and 2 drives running as a mirror and it will be much cheaper (~700-800USD).

Resources: http://www.raid-calculator.com/default.aspx

View on HN

Don't run 12TB of data with consumer hardware in a RAID5. If you have the money stick with RAID 1, 10, maybe 6 or even JBOD.

RAID 6 will be almost as slow and vulnerable as RAID 5. The best one in terms of speed and data security would be RAID 10 or RAID 100.

Gah! Why is anyone even uttering the word "RAID". Ugh. ZFS people. ZFS.

I agree, you can see from my other comments in this thread, I am a strong advocate for FreeNAS :)

What's the implication here? Do you eat the 50% storage efficiency with mirrored pools? Or do you use a wider stripe for better resiliency (raidz2, raidz3)?

I've got an 11-disk system with raidz3.

All the striped modes will (in effect) limit write rates to the bandwidth of a single disk, but ZFS is good about this: So long as the writes are reasonably large, as they'll likely be for a media server, you'll usually get the maximum streaming bandwidth of the disk. So, 150-200 MB/s. Reads are much faster.

It's not something you'd want to use in a DC, but for mostly-idle storage at home it's perfectly fine. Add in an SSD cache, and it'll be downright speedy.

That gives an effective capacity of 64 TB, given 8TB disks, and the chance of four disks dying before you can replace even a single one is pretty low. The usual limitation of ZFS applies, however: You have to decide the number of disks up-front, as you can't expand the size of a RAIDZ vdev later on.

(Well, you can, but only by replacing all the disks with larger versions.)

I run stripped mirrors with ZFS. 2x2. RAIDZ is simply to slow in comparison. Like A LOT slower. Ill just eat the storage loss with mirrors. Disk is cheap.

Disk is cheap, but how you attach the disks starts becoming an important cost component as you add more disks. :-(

IOPS aren't cheap, and you will find out just how much that matters the moment you have to replace a disk in a pool with a lot of data in it. Resilver times of days can be pretty agonizing in a home setting.

I won't build a pool without at least double redundancy per rotating-drive vdev, meaning at least 3-way mirrors (zpool create foo mirror disk0 disk1 disk2 mirror disk3 disk4 disk5) or raidz2, as I have had to go to backups for failures during resilvers of raidz (never again!) and 2-way mirrors.

3-way mirrors conveniently provide high read IOPS. OpenZFS does a good job aggregating and scheduling writes to the point that writes are mainly choked by random reads. (Very full pools induce quasi-random reads while hunting for space in metaslabs, which is the main reason writes to very full pools perform poorly).

2-way mirrors are only really fine for vdevs that have negligible random-access penalty (and thus likely high IOPS), or where rebuilding an entire pool from backups takes time comparable to resilver and scrub times when dealing with a replacement leaf device that might fail during resilvering (you cannot know the data on the replacment device is OK until you have read it, which generally means scrub-after-resilver), or errors reading the surviving leaf device(s).

RAID is fine. As long as it's RAID 10 haha.

What's your reasoning behind that? I would trust a 6+2 RAID 6 more than a 4+4 RAID 10.

If your that concerned with redundancy use a 3-disk striped mirror with ZFS. Redundancy AND great I/O.

Resilvering time. You are faster getting a broken drive fixed from a mirror. This gets more important with increased drive capacity.

There is an article somewhere calculating the probability of two drives failing during resilvering and how this gets worse with the ever increasing capacities, which increases the resilvering times.

But I guess you already account for this by using a Raid 6 with +2.

I can't think of any reason resilvering should be significantly slower with RAID 6. Even an extreme bottleneck of attaching all the drives to a single SATA channel would let you resilver a 10TB disk in roughly one day.

Triple parity can give you pretty good integrity, but at a certain point it's wasted effort. Disk failures cluster too much for any single array to be safe; you need backups.

That's good advice. I was trying to cut the price down a bit so OP didn't fall out of his chair. Maybe 10TB was an unlucky number plucked out of the air and just two 8TB drives running in a mirror will suffice.

I am running raidz1 on a 5-disk home fileserver with the philosophy that if I do get bitten by a multiple drive failure or latent bad sectors during a rebuild, I can count on zfs to tell me which files were affected and then I'll just restore them from backup. (Unlike md raid where rebuilds seem to be much more all-or-nothing.) My underlying assumptions here are:

* A disk failure where some moderate number of sectors become unreadable is far more likely than a disk going completely toast.

* Even if I were running raidz2 or raidz3 I'd still want the offsite backups in case of some disaster befalling the entire array (or more likely, me accidentally rm'ing something)

* It's a home fileserver used mostly for archival purposes. Unlike a business scenario, the loss of availability while I restore from backup is not going to hurt too badly.

Did I miss something here or do you think this is sound?

Looks good to me.

>You probably need a NAS. To store 10TB securely you are looking at a bill of materials of about ~1200usd.

For just 10TB, I would recommend that most home users stay away from RAID and buy 3 Seagate 10TB drives (street price currently $399 each) for total of ~$1200. Use 1 drive as primary copy and the 2 others as backups. With 3 drives, you could even rotate the 2 backups to an offsite location.

I quit using RAID in 2007. All of them FreeNAS/drobo/Synology/etc required more troubleshooting and babysitting than I wanted to deal with. It didn't matter if it was Linux software or dedicated hardware (NAS appliance).

The advantage of a NAS is that the file system works equally transparently on OS X, Windows, and Linux. Attached drives can't do that. (Not easily and reliably, anyway.)

You may not need that feature, but it's a very useful option to have.

The downside for media storage is that a straight NAS is often incredibly slow compared to USB3. For decent speed you need iSCSI, which adds cost and complication.

>"For just 10TB, I would recommend that most home users stay away from RAID and buy 3 Seagate 10TB drives (street price currently $399 each) for total of ~$1200. Use 1 drive as primary copy and the 2 others as backups. With 3 drives, you could even rotate the 2 backups to an offsite location."

And how do you plan to detect if your data is silently corrupted? Unless you're using some archive file format that includes checksums/hashes, you have no way of knowing whether your backups are any good until it's too late.

Your tone makes it sound like silent RAID corruption doesn't happen. That's not true, even for NAS appliances like Synology.[1]

If you meant that a filesystem like ZFS with checksum blocks is designed to protect itself agains corruption errors, one can use that without RAID. A single-disk ZFS is orthogonal to RAID.

To point back to the OP's question, he's running Windows with Plex. In that case, a 10TB NTFS filesystem with periodic binary comparisons/checksums (or format the disk with the newer ReFS[2] that stores checksums similar to ZFS) is simpler than RAID5/6. He also gets the extra benefit of an offsite backup without paying for cloud storage.

I'm not against RAID for all cases but a home media computer with just 10TB doesn't meet the threshold of dealing with its extra complexity. However, if one is wanting to consolidate 25TB+ of disk space -- or running a business -- or need aggregated io bandwidth, the cost/benefit drivers for RAID over JBOD would make more sense.

[1]https://www.google.com/search?q=raid+data+corruption+synolog...

[2]http://www.windowscentral.com/how-use-resilient-file-system-...

I built a home backup server that works good enough for me:

* zfs for snapshots (and send my inc snapshots from my laptop) * zfs fs with snapshots to rsync non-zfs enabled devices (read: wife's mbp && imac) * rpi3 as brain (slow, but works) * 2x2Tb drives on a zraid

It took me a while to get it to where I wanted, but it was cheap (~230), low power and safe enough for home. I recently did a full reinstall from scratch and it took me less than 15 min to get my system back (I have my /home as a zfs fs backed up).

What are your components???

I would second going with something like a synology NAS vs a PC box with freeNAS mostly because of power consumption. A cheap PC box has about ~60W idle when just sitting there doing nothing as a server, while the NAS has 10-15W idle.

What is your strategy if the NAS (controller) dies in you and you need the data? Is Synology using proprietary formats or could you just put the drives in a desktop machine? Is the strategy to buy a new Synology? Or do you treat the NAS as a backup and add an offsite archive (e.g. S3), and then you can restore the archive to any device you want (new Synology, freenas).

This comment should not be understood as offensive. I am just interested where one draws the line.

From what i read on the Synology website they use EXT4 or BTRFS for the internal disks; so no proprietary format.

Synology's NASes uses Linux mdadm, so you could use either a replacement Synology NAS, or a PC [1] to access your data.

[1] https://www.synology.com/en-global/knowledgebase/DSM/tutoria...

You can restore drives from a dead synology like other people said, and I also have a amazon cloud drive backup. The NAS is for fast onsite backup. I use arq to backup to both destinations.

You always want 3 copies: one on your computer, one onsite and one offsite. If there is a fire or everything electronic is stolen from your house, you still have the offsite.

I've been very pleased with my Synology. I have a two bay with 1TB in RAID1. I will upgrade next year - probably will get a DS416 with four Red 6TB in RAID10.

I just put a Synology 1815+ together, with five 4TB Hitachi drives (shout-out to Backblaze for the recommendation). Using Synology's 2-drive redundancy option I have about 10.5TB available and 3 bays free for expansion.

The only hassle so far has been backups. The built-in backup apps can't span media (external USB drives). Something I used to do decades ago with 3.5" floppies... So I have to do a file-copy "backup" across the network.

Reply on news.ycombinator.com