Behind AWS S3's Scale
highscalability.com
highscalability.com
1956: a 3.75MB drive cost $9k
2024: 26TB drives exist, where 1TB costs $15
It apparently cost $3200 per month to lease one of them [1] so actually a storage payment model akin to S3...
[1] https://www.dataclinic.co.uk/history-snapshot-1956-the-world...
Everything is recorded digitally online, even the signage is becoming digital - soon all evidence of our language and technology will be tiny and invisible, the paint on the road, our clothing and accessories, compressed into hard drives and displayed on screens. Thousands of years in the future, if someone stumbled upon the ruins of our cities and discovered a smartphone, I wonder if they'd be able to figure out what it is. There'd be no obvious controls, too small for storing anything meaningful, too fragile to be a weapon or tool. Even if it was possible to read the values on the hard drive, and figure out that it was a computing device, would the values make any sense? Would they be able to figure out that the meaningful data is on some other hard drive at various data centers around the world?
Who knows what other invisible magic is out there from past civilizations. In 50 years a computer went from being science lab to a pocket watch.
Whenever people surmise that maybe past civilizations were high tech but we just don’t see evidence - we absolutely would see evidence, because the reason we have a high tech civilization is that there are enough humans to have one. Think about the Sumerians - their civilization by some measures lasted for nearly 4000 years, BUT at any given time there were never more than perhaps half a million people living. And a typical city would have had a few thousand people (a huge one like Babylon, maybe 100k).
That kind of population isn’t building integrated circuits. It’s big enough to support enough copper merchants that some of them are terrible, but that’s about all.
It's infrastructure, too. Some people think the "Dendera light[0]" is an actual ancient Egyptian lightbulb but never bother to wonder why there are no sockets or wires at the Temple of Hathor for it to draw power from, or where the power source was, or where the switches were, or why the Egyptians apparently only had one lightbulb to their name, why the Romans never mentioned the Egyptians had electric lights, why they saw fit to carve a picture of a lightbulb onto a temple wall, etc.
>It’s big enough to support enough copper merchants that some of them are terrible, but that’s about all.
A million years from now when humanity has merged with the Matrix and spread across the universe as beings of pure thought... poor Ea-Nasir will still be known for his lousy copper and customer service.
What I meant by ancient invisible technology is more like... consider a USB stick. What about that physical object would reveal what it is if you strip all cultural context (including language) away?
When we see depictions of some ancient symbols (e.g. those handbags and wristwatches all over the ancient Middle East and Africa) I sometimes wonder if it is simply an unknown technological device - we'll never figure it out being in our time and place.
Historians think everything is a temple or a tomb, instead of more everyday things like drive-thru restaurants and gaming devices. Surely those existed too, and if they did they probably iterated on them - they probably had some innovations we haven't thought of and wouldn't recognize today.
Every call is authenticated. Changes to authorization ripple across AWS in realtime. If you revoke a priv, things stop working immediately. That's incredibly hard to do, especially when you're authenticating billions of requests a second.
For billing and telemetry, everything's is logged. There are companies that are built on the idea of logging, and at AWS it's just something they do - without slowing anything down.
AWS just might be one of the most complicated things humanity has ever built, which is a weird thought.
for a human-scaled definition of immediately, perhaps, but IAM updates are eventually consistent across the globe.
This is also the “power” behind snowflake and bigquery.
Also happy that highscalability is still up and about.