Hacker News Reader

Comment by 0xCMP | Hacker News Reader

The unfortunate part of that graph is that I think it's not updated for today's prices given the memory shortage/crunch we're experiencing that's driven up the prices for all kinds of memory.

But also while it should be faster than it is, how would an S3 designed around SSDs look differently to an API user? I would think the API is basically the same.

View on HN

That's S3 Express One Zone. Directory buckets are SSD-backed and regular buckets are HDD-backed. Some of the API differences off the top of my head:

- Directory entries are no longer returned in sorted order in ListObjectsV2

- There's an AppendObject API

- There's a RenameObject API

It's hard to imagine how these API differences can be explained by the different underlying block device. I don't see any good reason you couldn't support these operations on a HDD.

I suspect it's more to do with the fact that with One Zone is a clean rewrite of large parts of the application stack that makes up S3.

S3 is made up of hundreds of microservices[0], there probably isn't anyone at Amazon that actually understands the whole system. Refactoring it to support these features probably requires coordination between a lot of different teams. They might have petabytes of metadata, making a change to how metadata is persisted probably requires a massive risky data migration.

[0]: "All in, S3 today is composed of hundreds of microservices" - https://www.allthingsdistributed.com/2023/07/building-and-op...

For what it's worth I think you are correct, and that these differences have nothing to do with being SSD-based.

It would probably look a lot like existing S3 concepts that have explicit hot and cold tiers. For example Intelligent Tiering, or Glacier.

Reply on news.ycombinator.com