Diffoscope – In-depth comparison of files, archives, and directories

When I was working on Othernet (www.othernet.is), I wanted to do Over-The-Air updates of our firmware. Over a unidirectional, slow (~20kbps), broadcast satellite link with significant packet dropout this was quite a challenge. Amongst the many tools that helped in making it work was diffoscope. I used it to make our buildroot-based firmware have deterministic, minimal-diff build, thus reducing the amount of update I needed to send out.

For a few weeks, I simply iterated between building the firmware, using diffoscope to diff the new build against a previous build to find what files changed and where, fix buildroot packages to eliminate these file changes whereever possible, lather-rinse-repeat.

In the end I was able to reduce our OTA update sizes to a typical 20-50KB (base, compressed firmware image as 30MB).

Fun times!

It's surprising how often projects retag releases. Diffoscope makes it easy to see the difference between the original tarball and the retagged versions.

Changes vary from from forgetting to update an in-source version to tagging the wrong commit to fixing file metadata (file owner in the tarball).

If you want to try this without any setup, https://try.diffoscope.org/ is a nice web interface to it.

It looks like it uses a Wagner–Fischer algorithm to find the diffs. I think it would be nice if that is mentioned in the documentation. In formats that are not order dependent the algorithm can't find a good comparison because no generic algorithm can without more input.

Diffoscope is used extensively by the Debian reproducible builds project to compare software build artifacts and find how they differ.

nice, i like the tree view.

I have been using beyond compare for this sort of thing for longer than i can remember though.

Edit: actually, bc has different features.

Meld can also compare directories