Hacker News Reader

Comment by Bizarro | Hacker News Reader

One tool that I would like (and is probably) out there is tool that would easily let me compare and merge directories. I think this goes beyond Beyond Compare. But here's my use case:

I have videos, images of our children in various places (my phone, one drive, dropbox, maybe usb sticks) that I want to go into the target directory (probably one drive). the tool would go through all the other directories, checksum them against all the other directories, and then present them as "these are the files that should be merge into your target directory".

Can regular diffing tools do something like that in an easy manner?

View on HN

Maybe off-topic to your original need. But I've always wanted / needed a utility that can de-duplicate media (images, music, videos) that I have scattered around which are the same thing just at different qualities/resolutions.

I get that for music and video this is actually quite challenging but images - - I would've thought quite easy? I've tried a few tools which claim to do this over the years but they all fail rather miserably / require a paid upgrade to do it better (although what is demonstrated in the free version is never impressive enough to consider paying).

Might be a good project for myself in my spare time, but wondered if anyone knew of anything obvious I was missing? FSLint and stuff is great for general filesystem de-duping, this is a slightly more complex case.

You might want to try digiKam[1], it has a fuzzy search feature for finding similar images[2].

1: https://www.digikam.org/

2: https://docs.kde.org/trunk5/en/extragear-graphics/digikam/us...

I work a lot with image and video processing - this sounds like the kind of thing I'm interested in. Would you want to chat some time? Email is in my profile.

I use rsync for something similar to this; it might help you too. I use ‘-P’ which makes the destination be like the source (so you never accidentally delete files in the source) and --dry-run will list what changes would be made without actually changing anything, effectively givingg you a directory diff. I’m sure there’s a flag you could use to say “I only want to create new files in dest if they’re not in source, but not delete or change anything” but I don’t know what it is off the top of my head.

I use a tool that I bought called Resilio Sync that has good usability, but not the flexibility that something like rsync would have. Thanks for the advice.

rsync is definitely capable of doing what you want. Check out the man page and use '--dry-run' to see how each command would work.

Also, the -P option (at least on the version I have installed) is for enabling both --partial and --progress. --partial keeps partially transferred files in the event something goes wrong, and --progress shows you the progress of the transfer.

You will probably want to use -r for recursing into your directory structure.

Total Commander

Thanks, I'll take a look at that. I'm assuming it can recurse into subdirectories and you can use wildcards/regex?

Reply on news.ycombinator.com