Hacker News Reader

Comment by loeg | Hacker News Reader

SMR drives can be thought of as tape libraries. You have a bunch of large, independent tapes that can all be written sequentially. But you can't do random writes to the tape. (SMR drives also have a lower density, non-SMR "scratch region" that can be used for metadata, or to build up to a larger I/O piecemeal. My metaphor is breaking down here.)

They're not generally useful for the kinds of applications harddrives are classically used for (classic filesystems). They're mostly useful for applications that already use tape, with the benefit of much quicker random reads.

Log filesystems may be able to take advantage of them. There will necessarily be I/O overhead to garbage collect deleted files out of SMR regions. (Because you have to rewrite the entire SMR region to compact.)

Is anyone using or planning to use SMR drives in production? If you're able to share, I'd be curious to learn about your use case and how you plan to make efficient use of the disk.

View on HN

I use a bunch of 8TB SMR Seagate Archive drives in a ZFS pool for backups and media storage.

I run ZoL, and honestly they've surprisingly been relatively solid drives. You can expect roughly 10-15MB/sec or so write throughput per drive, and latency is of course pretty bad. Across 24 spindles though, I haven't had too many complaints - it's archival storage and reads are fast enough for most use cases - about 50-70MB/spindle sustained for large files.

I would not try to say rsync millions of tiny files against this pool - it would not hold up well. However for it's use case - write large files once/read occasionally I'm quite happy - I just wish price would have come down over the course of 2 years as I had originally expected. You can expect to pay $200-220/drive even today, and that's what I paid the first batch when starting my pool.

Out of 24 total spindles I had 2 early failures (within 120 days of install) but otherwise no other failures. These drives were replaced hassle free via RMA. My I/O pattern is pretty light - probably 100GB written per day across 24 spindles, and maybe 1TB read.

Basically if you try to do anything but streaming writes you're going to have a bad time. They are a bit more forgiving on the read side of the fence however. Don't expect these things to break any sort of speed records!

If I were buying today I wouldn't buy 8TB SMR - I'd pay the $20-30/spindle premium for standard drives. I'd have to look at the 14TB costs to see if the huge speed tradeoff would be worth it. When I first started using them, the cost per GB was compelling enough to give it a shot and I'm pretty happy with the results.

What do you need to do to get FreeBSD to use SMR drives properly with ZFS? Are there special drivers?

It seems ideal for a write-once/read-many media server.

These are drive-managed SMR, so no special drivers needed. I agree they are ideal for write-once/read-many media servers :)

Today I'm not sure the 8TB drives make any sense as prices have come down so far on regular 5400rpm slower drives that are much faster. These new 14TB spindles will be interesting to keep an eye on.

I imagine SMR won't really take off, if it does I'd expect more direct kernel/driver support for the hardware. Drive-managed SMR is always going to be exceedingly inefficient.

SMR drives have been shipping since 2014 (eg Seagate Archive HDD), and people are using classic filesystems with them to replace media storage, backups, and other largish file apps. It's much slower than traditional spinning HDDs at small write IO's, once you blow the write cache layer, but not tape-level terrible. You wouldn't use one for your root FS. But then again you wouldn't use a spinning HDD there anyway.

There are ideas about "host-managed SMR" too where the HDD device presents the low level SMR device, and would need customized filesystems and drivers. But the idea isn't very attractive in the marketplace.

For sequential writes they (i.e. the Samsung Archive v2) are relatively decent (about 80-150 MB/s depending on the mood of the drive).

Supposedly Facebook is trialling them with cold storage -- think backups they must keep legally but don't intend on using much.

I could see a usecase for certain NAS / media storage purposes, but you'd have to not only use log structured filesystems, but also redesign the network protocols to support efficient writes, and possibly specialize block allocation to match the hardware constraints. You certainly wouldn't want say bittorrent writing directly to them, and even streaming-like services like MythTV may be a problem.

They are probably a good choice for timemachine like backups, where every write is incremental.

So as I understand the primary issue with SMR drives is that non-sequential writes tends to be slow because several tracks needs to be rewritten. Wouldn't the perfect solution be to make it a hybrid disk instead (i.e. add a small SSD to it)? That way the data can be written to SSD quick and then the firmware can move it onto the platters slowly.

Of course there are some issues about balancing SSD size vs. cost and write cycles, but at least if designed properly the failure mode once the SSD part is weared down would be that it just functions as a regular (device managed) SMR drive.

This can essentially be accomplished with ZFS and an SSD as a L2ARC device.

I have three 8 TB Seagate SMR drives in RAID-Z (aka RAID-5) with a 500 GB Samsung 950 Pro M.2 as a L2ARC cache drive on the pool and it works beautifully for my workload.

In my experience, once the scratch area of the SMR drive is full, I get about 4-5 MB/s sustained write speed, which in my pool translates to 10 MB/s for the pool. Since the scratch area is about 25 GB, that means I can do 500 GB + 40-50GB of random writes before things slow to a crawl, and I have to wait (550 GB / 10 MB/s) = 55,000 s ~ 16 hours for the writes to flush.

I paid $179.99 each for these. Not Too Shabby.

Sounds like they would be perfect for Cassandra, or anything else that uses SSTables.

Why? SSTables/LSM storage for file data is completely separate (and much higher level) from physical storage tech like SMR. These drives are very slow compared to even standard hard drives and probably cant even keep up with writes, let alone compaction.

Reply on news.ycombinator.com