Did you consider using the PhotoDNA hash algo for finding duplicates? If you’ve heard of it and ruled it out, love to know why. While designed for a very different (and dark) purpose, seems like it might do well for the task.
Just had a look, thanks for the pointer. And yes, it was designed for a dark purpose. Will try to find the comparisons with the currently implemented hashing methods and see if there's merit to implementing it.