Sad to see they going down this road. Any alternatives?
a local filesystem remote
a cloud provider remote
rclone serve s3 to serve a local s3 compatible API for the local remote
something orchestating an rclone copy, rclone sync or rclone bisync command to sync the local and cloud remotes. Could be triggered on a timer, or using another tool to watch the local directory for changes and trigger a sync.
I think you can run rclone as a service, they recommend you use bisync on crontab, for example.
What's bisync? I don't really know, as it's new to me, but it's an experimental beta feature from rclone and sounds neat.
https://rclone.org/bisync/#sharing-an-encrypted-folder-tree-...
> Sharing an encrypted folder tree between hosts
> bisync can keep a local folder in sync with a cloud service, but what if you have some highly sensitive files to be synched?
> Usage of a cloud service is for exchanging both routine and sensitive personal files between one's home network, one's personal notebook when on the road, and with one's work computer. The routine data is not sensitive. For the sensitive data, configure an rclone crypt remote to point to a subdirectory within the local disk tree that is bisync'd to Dropbox, and then set up an bisync for this local crypt directory to a directory outside of the main sync tree.
This is quite an amazing man page, at no point does it say what the command does.
It has a lot of details upfront about it being beta, and where its working directory is, and what the limitations are. But what does it do?
https://github.com/rclone/rclone
Specifically, check out this new beta feature called bisync which has use cases that sound a lot like yours:
https://rclone.org/bisync/#sharing-an-encrypted-folder-tree-...
Before investigating it for eventual production use, I heard quite a bit about how complicated it was to use, but for my use case (storage for enterprise and academic k8s clusters) it's actually been quite simple to deploy and use. cephadm (one of many ceph management tools) can handle nearly all our bootstrapping and management needs. Little to no tweaking or configuration needed. Fairly low overhead. Very reliable and resilient to adverse conditions. Easy to handle different storage types and data retention needs.
One thing I will warn you, if you go the ceph docs site right now and just start browsing, it is in fact quite overwhelming because ceph has the capacity to handle a ton of edge cases and unusual environments. I'd recommend taking 15 minutes and build this [1], which gives you a fully functional toy-sized ceph cluster on a single node. Then, hit the docs to fill in the gaps of what you need to know for your deployment.