Hacker News Reader

Comment by woah | Hacker News Reader

What is the mechanism that makes it cheaper to take longer getting data out? Is it that they save money on a lower-throughput interface to the storage? Is it simply just market segmentation?

View on HN

In theory, tape [1], optical [2], or spun-down disk [3] are cheaper but slower than spinning disk. Erasure coding [4] is also cheaper but slower than replication. One could even imagine putting cold data on the outer tracks of hard disks and warm on the inner tracks. In practice I suspect Glacier is market segmentation.

[1] http://highscalability.com/blog/2014/2/3/how-google-backs-up...

[2] http://www.everspan.com/home

[3] https://www.microsoft.com/en-us/research/publication/pelican...

[4] https://code.facebook.com/posts/1433093613662262/-under-the-...

it has more to do with clever uses of erasure coding than you might think.

Pure speculation here, so I'm probably completely wrong, but I imagine it's so that they can essentially reduce the amount of seeking. Glacier uses magnetic tape storage, I think, and I believe each tape has to physically be inserted into a machine to be read, and then be removed afterward. So there would be some downtime as tapes are swapped in/out. Therefore, it would make sense to aggregate reads ahead of time and maybe even physically reorder whole tape accesses to reduce the time it takes to load them.

But this wouldn't explain why the read rate factors into cost. Maybe they scatter data across tapes as well, and higher bandwidth requires loading more tapes concurrently?

Again, total conjecture. Please let me know which parts are wrong and which are right.

It is rumored that Amazon denied that Glacier uses tape. It's still not that clear what it does use. https://storagemojo.com/2014/04/25/amazons-glacier-secret-bd... (also read the comments)

It looks like they're workload optimized drives.

https://news.ycombinator.com/item?id=4416065

Storing the data in tape systems or some other system which is automated but not available in a matter of milliseconds.

Tape is cheaper than hard drives are cheaper than flash drives are cheaper than ram.

Reply on news.ycombinator.com