Semhash: Fast deduplication and dataset multitool in Python
minishlab.github.io
minishlab.github.io
today we released Semhash! Here's a blogpost on how it works. We did a show HN yesterday, but that got deleted for some reason.
Let me know what you think!