Visualizing Entropy in Binary Files | Hacker News Reader

Visualizing Entropy in Binary Files

corte.si

47 pointsjgrodziski12 comments

Author of the post here. I'm working on a browser-based interactive analysis tool based on these visualizations. It's nowhere near ready for release, but those who are interested can find a demo here:

http://binvis.io

This is not even an alpha yet - I plan to announce the initial release in the next month or so. Eventually, I would like to add the ability to upload, share and annotate files for analysis. Comments and criticisms welcome!

The level of detail and structure you can pick out with this tool is amazing.

https://i.imgur.com/lsRA1RN.png

Removed?

That's excellent! Would be very cool to also have frequency digraphs for the current segment.

Frequency analysis and byte dot plots are definitely on the way. Do you have a concrete example of the digraph frequency feature you can link me to? Any other feature suggestions are welcome too.

The link below to the Cantor Dust presentation shows 2d and 3d examples. I wonder if there are advantages of other space-filling curves, like http://en.wikipedia.org/wiki/Sierpi%C5%84ski_curve and http://en.wikipedia.org/wiki/Peano_curve

Right - these are the frequency dot plots I first saw in Greg Conti's work (some examples here: http://sebug.net/paper/Meeting-Documents/BlackHat-USA2008/BH...). They're definitely on the way.

Regarding other space-filling curves - I'm in two minds about this. I've implemented z-order and the H-curve for binvis, but for different reasons neither of these are as useful as the Hilbert curve, so they're not exposed in the interface. My feeling now is that I should pick one locality preserving layout and one intuitive layout to keep things simple and usable. I want binvis to be a workaday tool, so I'm keeping my space-filling curve geekery to another project (https://github.com/cortesi/scurve).

Have a look at Cantor Dust [1], a visual reverse-engineering program that is promising.

The author says in a presentation [2] that CD is based on Cortesi's work.

[1] https://sites.google.com/site/xxcantorxdustxx/visual-re

[2] https://www.youtube.com/watch?v=4bM3Gut1hIk

Here is a demo: https://media.blackhat.com/bh-us-12/Arsenal/Domas/_cantor.du...

These images remind me of Piet[1], an esoteric programming language where the programs are images. There you see the structure of the algorithm, rather than the binary, visually presented.

[1] http://www.dangermouse.net/esoteric/piet/samples.html

This reminds me of a tool that came out 10 or 15 years ago, which allowed you to feed it arbitrary binary data that it could display in 3D.

I remember reading a Slashdot article on it in which one of the examples was a 3D, freely-rotating view of the Linux kernel binary.

Does anyone know what I'm talking about? I can't seem to recall the name of the project or find it.

Also related:

http://vimeo.com/110257380

http://gynvael.coldwind.pl/?id=199

https://www.youtube.com/watch?v=5f7hZBoEwV0

https://github.com/pwaller/binview

Perhaps a naive question but... what is this useful for?

reverse engineering binaries. as patterns are visualised you can also skip reverse engineering for some goals. for example general entropy levels of bytes (in relation to their siblings) may allow one to quickly pull out encryption keys from binaries or memory images.

Reply on news.ycombinator.com