S3 is an engineering marvel, but it's an insanely complicated backend architecture just to store some files.
S3 is an engineering marvel, but it's an insanely complicated backend architecture just to store some files.
One server with a RAID array can survive, usually, 1 or maybe 2 drive failures. The remaining drives in the array will have to do more work when a failed drive is replaced and data is copied to the new array member. This sometimes leads to additional failures before replacement completes, because all the drives in the array are probably all the same model bought at the same time and thus have similar manufacturing quality and materials. This is part of why it's generally said that RAID != backup.
You can make a local backup to something like another server with its own storage, external drives, or tape storage. Capacity, recovery time, and cost varies a lot across the available options here. Now you're protected against the original server failing, but you're not protected against location-based impacts - power/network outages, weather damage/flooding, fire, etc.
You can make a remote backup. That can be in a location you own / control, or you can pay someone else to use their storage.
Each layer of redundancy adds cost and complexity.
AWS says they can guarantee 99.999999999% durability and 99.99% availability. You can absolutely design your own system that meets those thresholds, but that is far beyond what one server with a RAID array can do.
I'll note that data durability is a bit of a different concern than service availability. A service being down for some amount of time sucks, but it'll probably come back up at some point and life moves on. If data is lost completely, it's just gone. It's going to have to be re-created from other sources, generated fresh, or accepted as irreplaceable and lost forever.
Some use cases can tolerate losing some or all of the data. Many can't, so data durability tends to be a concern for non-trivial use cases.
Standard RAID configurations can only handle 2 failures, but there are libraries and filesystems allowing arbitrarily high redundancy.
As long as it's all in one physical location, there's still fire and weather as ways to immediately cut through all that redundancy.
Much like twitter it's conceptually simple but it's a hard problem to solve at any scale beyond a toy.
Some details in https://www.allthingsdistributed.com/2023/07/building-and-op... / https://www.youtube.com/watch?v=sc3J4McebHE
The long and short of it is that getting anywhere near the redundancy, reliability, performance etc of S3, you're spending A Lot of money.
They also already have support for storage tiering, replication, encryption, ACLs, integration with other services (from web access to sending notifications of storage events to lambda, sqs, etc). Uou get all of this whether you're saving 1 eight bit file or trillions of gigabyte sized ones.
There are reasons why you may need to roll your own storage setup (regulatory, geographic, some other unique reason), but you'll never be more economical than S3, especially if the storage is mostly sitting idle.
It's more costly immediately. S3 storage prices are above what you would pay even for triply redundant media and you have to pay for data transfer at a very high rate to both send and receive data to the public internet.
It's far less complicated though. You just create a bucket and you're off to the races. Since the S3 API endpoints are all public there's not even a delay for spinning up the infrastructure.
Where S3 shines for me is two things. Automatic lifecycle management. Objects can be moved in between storage classes based on the age of the object and even automatically deleted after expiration. The second is S3 events which are also _durable_ and make S3 into an actual appliance instead of just a convenient key/value store.
I’ve paid pennies a year to store data in s3 for the better part of 5 years. Can’t even buy a hdd with what I’ve spent on s3.
I don’t think it’s so easily observed at scale though, because at that point it’s hardly just the hdd cost anymore. It’s the hdd, server/compute, cabling, cooling, power, facilities, security, cabling, maintenance.
The TCO of data storage isn't just the drive - it just so happens that it’s still less than the cost of a drive up to some threshold. I don’t know if anyone having done a full cost model comparison. Everything I’ve ever seen assumes the data center is free.