Not what you asked for, but related to the method you described:

"Image Retrieval Based on Using Hamming Distance"

https://www.sciencedirect.com/science/article/pii/S187705091...

EDIT: Here's a python package that does image deduplication. Not ImageMagick and no idea on its performance though.

https://idealo.github.io/imagededup/