Show HN: GitHub – I tried to build AWS S3 from scratch | Hacker News Reader

Show HN: GitHub – I tried to build AWS S3 from scratch

github.com

21 pointsjohnsmith20769 comments

Over the past few years I’ve been thinking about how I could build SaaS and deploy it on my own infrastructure without needing to use any cloud platforms like AWS or GCP. In this repo I document my progress on building a clone of AWS S3 that functions the same as S3 using an exclusively open-source technology stack.

Open articleView on HN

Wow, I already knew about testing-stack stuff like localstack (different purpose and unrelated to this project), but the fact this is compatible and connectible with standard AWS client libraries is cool. Imagine self-hosting your own S3! That's already a software-app-inclined homelabber's dream.

I'm glad you like it, compatibility with the official client liberys was a key objective for this project otherwise its just a NAS with a web interface. I've been told there are some homelab subreddits that would really like this project but I don't use that platform, feel free to post there if you think it would be of value, I will be working on this for the next few weeks

you might be on to something with this self hosting cloud services. this could be huge!

I think this is pretty neat. Even if you don’t use it for a SaaS company I think it’s great to try your hand at making a known product.

This may also be something fun for late-college-students to do in order to brush up on their skills and to dive deep into tech that they may use in the future.

Thanks, yes it's definitely not meant to be a product, I just made it because I had lots of questions about how S3 worked under the hood. I'm still finishing the documentation but a college student could just follow the commands line by line and they should end up with the same system, it wasn't easy to get running

One of the hardest parts to achieve in building your own S3 clone might be 99.999999999% data durability and other SLA features.

A look at the S3 documentation shows that there is a fairly ludicrous amount of functionality atop the basic Hadoopiness of it all. The .pdf User Guide weighs in at a svelte 3,374 pages.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcom...

I would love to take a look at this; an S3 alternative and open source is a magic combination. Even if it's going to be a one-time product like once.com, I wouldn't mind that at all.

I actually build systems like this for clients for a one-time fee (see UpWork.com link on my GH profile), but all of the set-up instructions are in the repo free of charge. Just be aware even with my notes building one of these things isn't easy

Reply on news.ycombinator.com