Vectorized Emulation: fuzzing at 2 trillion instructions per second
gamozolabs.github.io
gamozolabs.github.io
This tooling is specifically designed for "hard" targets, while "hard" is subjective, think targets with fewer than 2 CVEs a year. Where getting even a null deref is hard.
I have used this on some soft targets and it's just as if you ran AFL against it, candy everywhere. The upside is that this tool usually "finishes" in an hour (no more coverage, no more crashes). Making it a bit easier to develop mutators/generators as you can run them to completion faster and have a more effective development cycle.
Correct me if I'm wrong, but he's trying to emulate 16 systems in parallel by vectorizing the instructions.
Ok, but this assumes that all paths are identical. Once you start fuzzing by varying their input, the paths will all vary at which point you're down to 16 non-identical paths again.
The whole point of fuzzing is to expose the different paths and this would fail horribly for that, surely?
Here he is simply trying to fuzz faster by executing vectorized code instead.
Any kind of software with a message or input processing loop will naturally re-converge at the top of the loop. It's quite neat.
In a typical message handling loop, the big-switch will immediately jump off to separate message-handling code, immediately wiping out the possibility of parallelisation.
No need to parallelise at all, the compiler gives you that information.
What you have seems like a neat trick, but not an efficient way of actually fuzzing.
It seems like he's fuzzing machine code. So he doesn't have the original source code.
That's why AVX-512 is essential: it contains mask registers that make this approach practical.
Each core gets a completely unique fuzz case, where each lane of the vector gets a small mutation. In _many_ cases this mutation doesn't even affect flow (eg, the mutated parts are skipped over or never parsed due to errors). Meaning all 16 run to completion. What's really important here is that when the small modification you made to an individual lane does actually cause it to diverge, you now know where and when that part of the input is used in the program. This information is huge and can be used to tweak weights and other parameters of mutators/generators, making them learn which fields to use when and how often.
With some better logic (covered in a later blog) I'll talk more about handling fully divergent cases by having graph analysis to find post dominators in functions and run VMs until they can sync up. Rather than the current model of "sync if you can", this will be a smart forward-looking sync that will ensure that by the end of every function all VMs will be running again (even if that means I have to insert artifical post dominator nodes to graphs).
All fuzzing I have done (with AFL and the like) the paths vary wildly and will end up in completely different parts of the stack.
Surely the advantage you're gaining by parallelising is completely wiped out when you lose sync (which to me must be most of the time).
I just can't see how you can possibly keep sync between parallel runs in anything but the most trivial application-under-test.
To me, it seems that this will necessarily degrade to a single path being active. Have you done any analysis on how many paths are active simultaneously over a non-trivial run?
What happens if the executable already contains AVX512 (or other SIMD) instructions?
Though I guess you can rewrite them into their scalar equivalent first, and convert that into AVX512.
This is done however not due to a theoretical limitation of the tooling, but rather a practical one in that I haven't written SSE lifters and decoders yet. If I were to write these it would work just fine.
However if I were to lift AVX instructions I would lift them to their scalar counterparts in my IL (eg. by emitting 16 32-bit operations). Lets say there are 16 `vpaddd`s in a row, thus creating a 16x16 matrix, when I lift it I'd lift these adds to 128 individual adds, and then generate 128 `vpaddd`s. While I'm doing 16x more the `vpaddd`s I'm also running 16x more VMs than the application originally had, thus it cancels out, and the net effect is that I'm still running `vpaddd`s as fast as the CPU can execute them. Technically in this case my stuff might even be faster as it's going to be 128 adds instead of 16, which means the CPU is running based on instruction throughput longer and decoding the same instruction and keeping latencies down.
However it's possible that this transposition of the matrix might increase dependencies between instructions or something of the sorts which might slow down performance.
That being said, AVX is pretty much only used for memory loads and stores in string operations in most programs anyways, so it doesn't matter much.
The masking stuff reminds me of how branching works (or at least used to work) on GPUs, so I wonder, could all this be made to work on a GPU too?
I'd love to lay my hands on a cheap 72x5 Phi based workstations...
But since Intel only sells those by the tray, I guess I'll have to wait until someone decommission a large supercomputer.