Storage orchestration layer presents many S3/blob providers as one S3 endpoint

been putting a lot of work into this since last time I shared it...a LOT has changed and improved. Have picked up some users and contributors, been finding and fixing bugs while improving/expanding functionality.

Great for homelabbers who want to take all the always-free tier s3/blob allotments and combine them together and lock them to the free tier quota to make sure you never incur costs. The compression also helps get as much as possible for free.

Great for zero-downtime cloud migrations.

Great to ensure data exists in multiple clouds when written with no complex scheduled tooling shit.

Pair a cloud backend with your local minio/seaweedfs/whatever and set replication factor to 2. Every object will instantly live in the cloud and on your local minio....if you local minio goes down applications will keep working because s3-orchestrator will fail over the reads to the cloud backend...when minio comes back it will make sure all data matches replication factor.

Would love more people to try using it and see what doesn't work or isn't ideal so I can continue improving it....my own use cases aren't going to expose to me everything somebody else might use it for. Also gh issues and PRs are very very welcome...just want to keep improving it.

Just point your applications/clients at s3-orchestrator instead of directly at a real s3 backend...clients don't have to know anything about s3-orchestrator or how it is actually managing the data.

Also, I setup a slightly better auth system for it recently and built a terraform/opentofu provider for handling users/virtual-buckets/permissions published to both public registries.

It also includes a little cloudflare worker that can be utilized to further eliminate costs for backends that are part of the cloudflare data alliance and egress on them routed through cloudflare is free.

shitty project webiste: https://s3-orchestrator.munchbox.cc