Off topic, any tool to deduplicate files across different external Hard disks?
Over the years I made multiple copies of my laptop HDD to different external HDDs, ended up with lots of duplicate copies of files.
Over the years I made multiple copies of my laptop HDD to different external HDDs, ended up with lots of duplicate copies of files.
There are a few different ways you could solve it but it depends on what final outcome you need.
I can't have like 10 external HDDs attached at the same time, so the tool needs to dump details (hashes?) somewhere on Mac HDD, and compare against those to find the duplicates.
cd /path/to/drive
find . -type f -exec sha256sum {} + | sed -E 's/^([^ ]+) \./\1,/' >> ~/all_hashes.txt
sort ~/all_hashes.txt > ~/sorted_hashes.txt
awk -F, 'NR==1{print;next} {print $0 | "sort | uniq -w64 -D"}' ~/sorted_hashes.txt > ~/non_unique_hashes.txt