> NF1 SSDs enabled an undisclosed maker of servers to install 72 of such drives in a 2U rack for a 576 TB capacity
That's 12PB in a 42U rack! I wonder what sort of storage densities Google, Amazon, and Dropbox are achieving.
That's 12PB in a 42U rack! I wonder what sort of storage densities Google, Amazon, and Dropbox are achieving.
These things look useful only if a tiny fraction of the stored data is hot, I imagine lower density that provides more flexibility in terms of processing room on the box, and networking around the box is probably a better option in a lot of cases.
Been pondering these devices for the past 10 minutes or so, can't think of a use for them except elaborate storage servers.
(Except maybe heat? I would assume these run at a higher temp than equivalent storage space in HDD, but I've nothing to actually base that on.)
If money's no object, presumably it's not "there's no use case for them", it's "there's no reason to do anything else"?
Even with HDDs, if you actually want to use all your throughput you're likely going to have problems with the bus (rarely needs to be optimised in 99% of cases) vs the drives (needs to be optimised in all environments).
On a dual socket Epyc system with 36 of these, each with a 4x PCIe connection 'only' takes up 144 lanes, leaving plenty of lanes free for even 8x Dual-QSFP+ 40GbE NICs.
Of course, I'm not sure how you'd physically pack all of that into even a 2U form factor.
Usefulness... data warehouses. If the price was right, we'd probably move to that to speed up various ingestion & reporting functionality. At 288TB (assuming RAID10) capacity, it'd be a perfect fit.
It's also PCIe 3 for now.
Still, I think it might be in a better position than Xeon for raw storage IO.
25/50/100 is cheaper/better on modern gear.
Update 6/22: Samsung made an update to its statements regarding the NF1 SSDs. The drives are based on the Phoenix controller, they do not use a PCIe 4.0 interface, but rely on a more traditional PCIe Gen 3 x4 interface.
On the other hand, he has a bunch of people researching the topics so that at least he knows what he's talking about.
He seems to have good clue about gaming hardware, which is the field he is targetting.
When you take random-access and make the data chunks big enough, it becomes sequential-access. Raw 8k footage is above this threshold.
But the main advantage of the SSD, IOPS, becomes less and less important as bitrates increase.
As you go bigger and bigger, using storage arrays no matter what drive tech, hard drives start to be able to perform perfectly fine at lower price levels, while also fitting much more data.
If you go with 1.2TB 750s... oh you can't actually fit 20-100 of those in a single workstation.[1]
So maybe we back off to a NAS attached over a 40Gbps cable.
40 of those hard drives will be a lot cheaper than 20 of those SSDs. If we grossly overestimate seek times, despite how big the frames are, we might estimate 125MB/s per drive. That's enough to completely saturate the connection. Oh, and the array has 10 times the capacity.
[1] 20 assumes a lower framerate and generous compression. 100 assumes 60fps 12-bit at a lossless 3:1 compression. If "raw" is taken completely literally then 10 hours of footage could require 300 of them (and some of the math would have to be redone).
Of course for video production you would choose slower, larger SSDs but still get an order of magnitude more performance, very handy when seeking through footage.
Being capped at the equivalent of 3 or 10 high-end SSDs isn't problem if there's no good way to fit more than that with any tech.
And you don't exactly need to stripe them all. You could set it up as a blob storage, maybe storing each blob twice so you can distribute reads better.
I know SSD is faster, but it depends on what the workload requires.
Then you need to shunt that back out over the network - using the same bus it just arrived on, most likely after a trip via the CPU to add protocol headers. With a 40 Gbit NIC the per-drive bandwidth drops to absolute max 204MB/sec and available network bandwidth per-drive is only 86MB/sec (22k IOPs), meaning at full whack and in the best possible scenario, and assuming no other overheads, you'd still only ever see 4-5% of rated performance. In reality and after accounting for things like packetization (and the impedance mismatch of 4k reads vs. 1500 byte network MTU) and the work done by the CPU it's probably safe to knock that number down by another 20-40%, depending on the average IO size.
Sure they'd kick ass for reads from a small number of clients and latency would be amazing, but it'd be so much simpler, cheaper, and flexible to just have the same disk split across 4 or more boxes. What happens if the motherboard dies? That's a megaton of disk suddenly offline, etc.
https://blogs.dropbox.com/tech/2018/06/extending-magic-pocke...
So Supermicro