Yes, I’ve been using it for about a year to populate databases with a reference DB dump. The current file is about 18GB. I use cloudflare R2 as the backing store so even though it’s being pulled very frequently the cloudflare bill is a few bucks per month.
Large files are good, but it may have performance issues with many (millions) small files.
Yes. And if you track transformations of the binaries or ml training
Would appreciate a good answer to this question. I deal with large medical imaging data (DICOM) and i cannot tell whether it's worth it and/or feasible.
It's very much feasible. I'm currently using DVC for DICOM, the repo has growth to about 5TB of small dcm files (less than < 100KB each). We use a NFS mounted NAS for development but the DVC's cache needs to be on the NVMe, otherwise performance would be terrible.
You should look at Icechunk. Your imaging data is structured (it's a multidimensional array), so it should be possible be to represent it as "Virtual Zarr". Then you could commit it to an Icechunk store.