Hacker News Reader

Comment by antongribok | Hacker News Reader

I actually ran into this last year. We ordered about 2 thousand 8TB HDDs last year for a Ceph cluster and had an agreement with our server vendor that they are not supposed to substitute the HDD model. Well, seems like they ran out and a small fraction of the drives were of a different, older model. The performance difference was 2.5x for our workload:

Older model: 670 IO/s and 2.9 ms average latency

Newer model: 1680 IO/s and 1.1 ms average latency

We got the vendor to send us new drives and shipped the older model back to them.

View on HN

Was the 'older model' in this case one of the models where it is known that manufacturers were shipping SMR drives without disclosing it?

No. Both models were PMR. I tried to find out more about what the differences are from the manufacturer, but didn't get far. Perhaps someone here can shed some light.

Older drive:

  model number: MB8000GFECR
  SATA Version: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
  ATA Version:  ACS-3 T13/2161-D revision 5

Newer drive:

  model number: MB008000GWWQU
  SATA Version: SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
  ATA Version:  ACS-4

My guess at the time was that the newer drive had a deeper command queue, but I'm not actually sure, and I think both drives had the same amount of cache.

Those model numbers seem to be the HP part numbers. Don't HP rebadge drives from other vendors?

If so, then it would probably be useful to see if the vendor drive model numbers are available.

For the dramatic increase in iops, I wonder if there are more head+actuator assemblies on the newer drives or similar? Along the lines of these:

https://www.seagate.com/innovation/multi-actuator-hard-drive...

These are both Seagate Exos drives, one is a generation newer than the other. I'm pretty sure both have the same number of platters, thus same number of heads.

And yes, HP changes the firmware and model numbers... super annoying.

I'm not seeing any especially obvious differences between those models (aside from the ones you listed). Were the drive versions mixed in alongside each other behind the same storage controller? If so, I would wonder if perhaps the controller's handling of different SATA versions played a role.

It's possible. Yes, the drives were sprinkled around in the inventory, and we had some servers where there was a mix of old and new drives in the same box, behind the same controller.

We use the controller in HBA mode, so the controller shouldn't be doing anything fancy, but you might be right.

I've seen controllers in HBA mode pull some oddball performance shenanigans when presented with differences as small as running different drive firmware versions across connected disks, so it really wouldn't surprise me. SATA drives have a pretty bad reputation for firmware releases introducing unexpected and undocumented behaviors. I used to be on the customer success team for a hyperconverged distributed storage product, and such things were the bane of our existence. Over the years the inventory of product-supported SATA devices shrunk considerably as a.) more and more quirks were discovered in products and b.) flash prices continued to drop to the point that spinning disks were in less and less demand, putting downward pricing pressure on SAS devices.

> had an agreement

>We got the vendor to send us new drives and shipped the older model back to them

not much of an agreement then, shows where the power lies

>MB8000GFECR

Seagate ST8000NM0055 at 2-4x the price

>MB008000GWWQU

another seagate

Reply on news.ycombinator.com