Congrats on what you’ve made!
What sort of noise are you running in to? (Curious)
In an old project we actually did something a bit more based on visual changes. Basically detected visual diffs and got coordinates of offsets to the change. We then found the smallest html container that encompassed the diff and highlighted it.
Using visual diff you can fuzz things a bit to handle artifact and small movements.
A good lib we have some miles on https://github.com/mapbox/pixelmatch
And this write up on niffy is pretty good