Western Digital: We Are Sampling 32TB SMR Hard Drives
anandtech.com
anandtech.com
If you're considering one, I would pay special attention to the buffer size, and ensure that all the transfers you want to do to or from the drive are significantly smaller than the buffer to ensure reasonable performance. That excludes most video too. Storj files are typically just a few megabytes, and typically arrive at a frequency of just one or two per second, which the drive can handle.
Weirdly enough, just using the tool instead of copying files with Explorer somehow stopped the weird hanging from happening, even without having to enable the actual throttling. Probably some bug somewhere along the driver/firmware stack triggered by the write block sizes.
Overall, I wish the drive vendors would expose some API to directly manage the SMR/CMR areas via software, just like the FLASH memory chips do. That would make the job of appending new backups + overwriting the old ones actually manageable with predictable and consistent timing.
They do, on drive models that are sold to the customers large enough to have the resources to re-write their storage stack to handle zoned block devices. The drives sold at retail will continue to pretend to be ordinary random-access block devices for the sake of backwards compatibility.
SMR drives aren't designed for having data shuttled between areas like that. They're meant to be used such that you write in long streams directly to the shingled areas. The slowdowns are clearly due to the abstraction mismatch getting in the way.
Maybe you can make it work in clustered file systems like Ceph if you make sure you have a big enough SSD-based write cache.
If they increased the size to 4TB say, to make it easier to import data to it, it might be worth it. But if it takes several days to write 32GB of data to it, it's not really gonna be worth it to most people.
I've wondered why WD/Seagate don't invest in a consumer solution where drives are literally just plug and play together and give you raid without the hassle of a raid controller or software. Eg. An 8TB drive connected to an 8TB drive is a mirrored raid. SMR drives would serve as "cold storage". Once written to, would never be rewritten again.
if you want that I think they do have in box solutions with two or more drives and a while back sold their own NAS drive system. Not sure what they have today but there are plenty of free or free enough solutions out there these days.
0: https://hackaday.com/2015/06/08/hard-drive-rootkit-is-fright...
I feel like 10gb superspeed hardware is more than a fast enough connection. The hardware is relatively cheap to build, it's just the software.
I'm not saying you need a super high speed processor or memory either, since the SMR drive itself can serve can serve as (slow) memory for the CPU. The trick is trying to figure out mostly how to replicate blocks across the raid, and then mapping the files to raid blocks.
They are wonderful for write once/read many storage of large files. Due the the sheer number of drives the write speed isn’t a major concern as it can saturate the 20gbps Ethernet bond to the machine.
Reads are the multi-GB/sec range though, so long as they are sequential.
They would not be my first choice, but for specific uses like backups and media storage they are a good value.
I got a Lenovo laptop for the parents a long time ago that had a hybrid hard drive; I think maybe 8 or 16GB builtin SSD. Man that thing was just absolute trash, so slow; ghastly long boot times or time to launch a browser.
Ideally a SMR drive has a SMR capable file-system, where it can write data into linear logs & them try to batch rewrite the metadata. Even the pathologically terrible random writes can be somewhat coped with via accommodating fs.
Linear writing and reading of 30Gb files is a great use case for it.
Edit: Actually I think we now have an equivalent of the DMCA so breaking the encryption on the Blu-ray would be infringing that. Oh well.
I assume it's because the actual read-write head itself must among the most expensive parts to manufacture, but is there no photo mask approach to this that can manufacture such a head array (or staggered series of them if there is some interference problem that prevents putting them side by side in a single strip) the same way but processor dies are made?
Because a typical drive has MILLIONS of tracks, and an RW head is quite wide, compared to its very narrow active area. It would not work.
Let's try some math:
A typical drive has 1.3 Tb/in^2 density. Assuming bits are approximately square (have a 1:1 aspect ratio), and the drive is locally linear, we can calculate that (ignoring ECC and metadata, which make the task even harder) that each square inch is 1.2e6 x 1.2e6 bits, meaning that you'd need a strip with a density of 1.2e6 read/write areas per inch. Assuming an outer platter diameter of 3 inches and inner diameter of 1 inch, you'd thus need this strip to be one inch long. So you need to engineer 1.2e6 RW heads instead of one and somehow fit 1.2e6 of them within one inch. I cannot even imagine how you'd do this, as each RW head needs to be a coil, and some way to get the data out (and possibly even a local amp)
Wouldn't that get you to your millions of heads of effective track coverage, without all the complexity of having to move two or three different hinges on several different actuators sweeping the entire arc? Naively speaking it seems like you could mount such a bar even closer to the surface of the track since there is no chance of a head crash, as it could be supported at both ends of its mounting.
Surely the entire armature that moves an articulated set of actuator assemblies plus the head itself would be heavier and more cumbersome and failure-proneto move than a linearly-fixed rail.
Obviously someone has thought of this before me so there's a good reason for it, but track density alone can't be it.
A strip of heads that move in unison won't be able to operate in parallel, for the same reason that on real hard drives heads serving different platters cannot operate in parallel without being mounted on independent actuators. So it seems like your suggestion only reduces the distance that any given head needs to move, at the cost of greatly increasing the mass of heads+wiring that needs to be positioned quickly and accurately. You've seriously inflated cost for no gain in bandwidth and likely no gain in latency (and even if everything worked out in favor, it couldn't do anything to improve rotational latency).
In this case, I don't know how the impression could be gained that I was suggesting mounting 100 or so individual assemblies of the very same read write head currently used on mechanical drives, each with its own separate wiring harness.
Obviously, yes that would substantially increase the cost and the mass and everything else, you're right. Thanks for setting me straight about the impracticality of very different solution than the one I described, which suggested the heads could be manufactured as a single integrated unit, the way every other matrix of active elements (OLED displays, memory cells, someone else mentioned mems, etc.) is made, which might require two additional wires for selection signalling, if that, depending on what could be multiplexed. In exchange for this weight and complexity you get rid of one, possibly two extra servo joints, one possibly two extra armatures, plus all the wiring and additional control logic required for positioning this very delicate articulated assembly.
Now, if this array cannot be manufactured because no current solid state process exists that can reproduce the characteristics of a wound coil, duplicated a hundred times in a linear array of cells on silicon or some other substrate, that's an obstacle I can reason about and accept as a plausible dead end.
https://pubs.aip.org/aip/apl/article-abstract/101/6/061909/1...
https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=91008...
Not mems but a good desc of the kinematics https://www.mdpi.com/2218-6581/9/4/99
The limiting factor is the fact that positioning the head on platter one to access track N does not guarantee that the heads on the other 8 platters are positioned over track N on their respective platters.
Nor indeed does each platter even have the same number of tracks at the same spacing in modern drives. Ever since embedded servo became the norm for hard drives (late 80s), there hasn't been a need to align each platter with the others.
Incidentally this is why a HDD with a bad head or media area will make clicking sounds, sometimes very loudly, as the head actuator slams against the stops since the controller can't see the signals it's looking for and can only sweep the surface in search of them.
Have there been attempts to use active cooling/heating (i.e. thermal sensors combined with piezoelectric effect elements embedded in the housing) to maintain a precise, consistent reference internal operating temperature, as a means of eliminating the need to accomodate thermal-induced drift in tolerances?
What you and others are describing is strikes me the way that opening up the package of a modern memory chip and finding a fabric of millions of tiny magnetic cores would, rather than a silicon array of capacitor/transistor cells - that for some reason no fundamental rethinking of the problem has prevailed and thus the only developments in RAM for almost half a century is miniaturization of what was being done in the 1960's.
I wouldn't call that "the same basic mechanism" as there have been lots of refinement over the years, but it turns out that having the heads themselves find what they need to read was the best solution.
Do you mean only one side (top/bottom) of one platter?
And I suppose you could use (a stack of) piezo motors on both ends to move the strip in x and in z.
They are actually not: https://news.ycombinator.com/item?id=39428728
That article provides the physical characteristics of a bunch of drives across several decades. The newest there is a 5TB drive that has tracks 85nm wide and bits 17nm long, so each bit is a rectangle with a 5:1 aspect ratio. The oldest is a 44.7MB drive with tracks 40um wide and bits 2.6um long, for an even more extreme ~15:1 ratio, but all the drives follow this pattern: the bits are much wider than they are long. The ratio for floppy disks is even more extreme; its calculation is left as an exercise for the reader.
Of course this doesn't change your argument that millions of heads would be necessary, at sub-micron spacing, to say nothing of how alignment could be maintained with continual thermal expansion and contraction.
It also seems possible that instead of a single super high density strip of heads which may be impossible to manufacture, that instead a series of much lower density array strips could be mounted in n ranks, each rank offset from the predecessor by n/(distance between heads in each strip).
I assume this is not an insurmountable engineering, but that someone has already had this idea or something similar to it, and after running the numbers found it doesn't deliver enough of an advantage over the conventional design to be worth exploring further.
Ehh a modern HDD is a marvel of precision manufacturing and engineering.
> rather than a bar fixed in place that spans entire radius of the writable surface, with an addressable strip of r/w heads that runs the length of it.
Your bigger challenge is getting a set of read/write heads that are across the platter able to maintain a consistent height/etc.
The closest I'm aware of, are the Connor 'Chinook' drives [0] that had two actuators, one on each side of the drive.
Broadly speaking, I'd ask if multiple heads too close throw off decades of head design experience...
[0] - https://en.wikipedia.org/wiki/Conner_Peripherals#/media/File...
A bar would need to be held at a fixed height above the surface. The flying head dynamically adjusts to both distortions across the surface, and temporal variations as the platter changes temperature. This height adjustment is entirely passive, dependant on aerodynamics rather than any electronics.
What I did not know is that this is still and effect that is relied upon in modern, ultra high precision, fixed medium drives.
That was called a fixed-head disk. They got rid of seek latency (still had rotational latency) and so there was a market niche for them as (e.g.) swap devices, but they had lower data density than moving-head disks did, so they went out of style. It looks like they are so obscure today that there's no wikipedia article, but here's an article from some other wiki:
https://gunkies.org/wiki/Fixed-head_disk
They are briefly mentioned in the general Wikipedia article on hard disk drives. There was also "drum memory" where there was a head per track on a cylindrical recording surface. Those seem to have dried up even earlier: https://en.wikipedia.org/wiki/Drum_memory
Here's an info page about the DEC RS03 and RS04 from the 1970s. The RS04 weighed 120 pounds and had 1MB of capacity.
https://www.storagenewsletter.com/2019/02/19/history-1974-de...
https://en.wikipedia.org/wiki/Delay-line_memory
https://www.rp-photonics.com/optical_delay_lines.html
( * For specific use cases )
There's probably an insane data throughput domain that requires autocorrelation filters that might benefit from an optic fibre bundle in which each fibre is slightly shorter than the previous one.
However, what's the reason that they are selling SMRs to the consumers? Price difference in stores is not that big between SMR and CMRs, and there are a lot of downsides.