I had been using Skylable's sx server for this but unfortunately it looks like they're ceasing operations, I'm in the market for a decent replacement.
I had been using Skylable's sx server for this but unfortunately it looks like they're ceasing operations, I'm in the market for a decent replacement.
If you upload by hash then you need your own ref counting for deletes (or just don't support delete ... storage is cheap after all!)
Serious answer though in that instance it comes down to already having a naming system in place that uses hashids to generate the filename. I could re-write the whole thing but it's not really a project that grasps my attention for long bursts. As it stands I've just moved the whole lot to s3 for now, even with it's larger footprint it's not much more in terms of cost - Mainly I use the image storage project as a way to test new object storage.
As an example off the top of my head I'm thinking block level de-duplication would be good for storing backup tars from multiple machines. In my previous post I was more hinting at file-level dedupe but that probably wouldn't do as much.
second, deduplication feature is great, for minio and S3 usuage. if the users decided to attach same file to multiple comments (in my case) deduplication would save me a lot.