Wraith – A screenshot comparison tool from the BBC
github.com
github.com
Maybe it's time to revive it.
I didn't put it out there because, although it's functioning perfectly, I'm not happy with the internals and would like to find time to refactor it before sending it out to the world.
You'll never be truly happy with the internals and, if you're like me, you'll be forever "1 final refactor, 1 extra tweak" away from perfection.
Example: https://github.com/cameronmcefee/Image-Diff-View-Modes/commi...
Information: https://github.com/blog/817-behold-image-view-modes
It is for regression testing of css modifications, whether from developer or simply when using some kind of compression tool on source code you know is correct.
There's a ruby refactor that allows you to just specify an environment var.
We use this for catching things that we'd never spot, a good example of how it has helped us is after a giant Sass refactor, we ran this tool against the old CSS site and the new CSS site, this showed us where things were off by a pixel or two (as well as big formatting changes).
We've found this tool rather valuable, we're not saying everyone will, or that others should use it. We open sourced it because it might help someone.
Any recommendations? I already know of OpenCV.
What I do is think of ways I can cheat. Sometimes an algorithm that looks really complex in mathematical notation boils down to a for loop for X, and a for loop for Y, iterating over every pixel in the image looking for some feature.
If you know where your buttons will appear, roughly, you can scan that rectangle in your double for loop until you hit a long enough sequence of red, then continue scanning for a sequence of yellow, then green. If you're looking for the edge of a window, scan for a solid line of the window border color.
Basically, think of the simplest possible way to express what you want to know. Your algorithm doesn't need to have a clue what a button is, or have any real intelligence. It feels like cheating, really, but it works quite well for simple tasks.
Good reporting on failed regression is almost as important as detecting them which is why I developed this on top of PhantomCSS https://github.com/Huddle/PhantomFlow
They even mention anti-aliasing rendering issues, and applying of a 20% fuzz factor to work around it...
Simplicity. This automatically tests the whole page "for free".
Robustness of feedback for time spent. A small visual change could cascade into lots of tests cases that must be updated (rendering is hard, look how long it took IE to get it right). You don't need to update any tests with this tool, you just give the OK to a visual diff with expected changes.
Dynamic content. The BBC homepage is going to be different every day. Writing tests to test this could be arduous and error prone. Testing production content on build 779 vs 780 is a simple way to test regressions.
Also, the difference between a wanted visual change and a break is dependent on the change you want. That makes this quite a flexible tool (it's not just "any difference is wrong") that can be used simply and is unlikely to have false negatives (while making it easy to ignore false positives).
Also, this means it can be verified by a non-technical person. All you need to do is verify that the differences are either 1) wanted or 2) irrelevant, and this is an extremely simple and robust way of achieving that.