Hacker News Reader

Comment by chrisper | Hacker News Reader

It's an SMR drive. So better for archiving than regular IO

https://en.m.wikipedia.org/wiki/Shingled_magnetic_recording for anyone else wondering

Not to be confused with ASMR: https://en.wikipedia.org/wiki/Autonomous_sensory_meridian_re...

If you are the target audience for these drives, you likely don't even consider regular (i.e. including random) IO as a use-case for modern HDD's . 10+ TB HDD's really only make sense for sequential IO, even if they technically still support random writes (e.g. the 10 & 12 TB PMR drives): the order of magnitudes of difference in random vs sequential IO performance make this a no-brainer.

If you look at the design of, for example, DropBox Magic Pocket, or Infinidat & Qumulo, you'll notice that their HDD access is really as sequential as possible. And if your storage layer is thus already optimized towards sequential writes, why not take the opportunity and get some capacity "for free" by adopting SMR drives?

I have a bunch of services running off a 8 12 TB IronWolf Pro array in RAID10, and it does pretty well, even for DB stuff.

Copy on write filesystems can probably be optimized for SMR by using TRIM commands to punch holes and rewrite the content sequentially in a new zone. Afaik both zfs and btrfs have plans to do this.

That way they can be useful for more than archival.

You probably mean log structured, not copy on write. CoW doesn't help make writes sequential, unlike log structured filesystems.

log-structured is a special-case of CoW; specifically it's CoW where the allocation strategy is sequential blocks.

I assume that the drive firmware remaps all your writes to make them sequential anyway for increased write performance.

For drive-managed SMR drives, yes, but these seem to be host-managed ones. So the filesystem has to be aware of the zones.

> ... the filesystem has to be aware of the zones.

Does that mean the drives simply will not work with an unaware filesystem or that it will work but performance will be poor?

They will not work at all. You have to issue special commands to the drive to be able to overwrite zones.

I was thinking that F2FS might be a good filesystem as a base on which you'd use an object storage abstraction layer (like ceph)...

However since I initially saw the news of these drives a few days ago Samsung also axed some Linux devs, which gives me pause and makes me reconsider the long term viability of this filesystem...

https://en.wikipedia.org/wiki/F2FS

A full blown filesystem is overkill for an object store. You could use something like libzbc ( https://github.com/hgst/libzbc ) to write directly to the SMR drives on the block level.

I believe Ceph now has abstracted the drives away through BlueStore, which simply puts a large RocksDB database on the drive, bypassing most of the functionality a filesystem offers. It should be much easier to make an SMR compatible version of the LSM-tree backend of RocksDB, than writing a full-blown file system.

It's not accurate to say that BlueStore is just a large RocksDB...

RocksDB is one of several possible backends for object maps. There is a lot more to BlueStore than just omaps.

Also, BlueStore was actually designed with SMR drives in mind, however certain components of it are best placed on solid state media.

Still seems random access might be an issue. But would love to see how eg nilfs2 (maybe on top of software raid) benchmarks against zfs on these big drives.

Depends... my NAS is mostly used for video/audio archives and generally only one device in the house using it, playing back a single file, or listing a directory. Would probably be fine in that use case. Beyond that, it's used for backups.

Seems like a decent use case. I'm not sure how well they'd work in a NAS with Raid-5/6 though. I'd been considering a new nas with 4-6 drives at 8-12TB already. Random I/O isn't my primary use, and I'm sure there are others at these sizes.

Reply on news.ycombinator.com