As someone who is actively developing a decompiler to reverse engineer old DOS 8086 video games, I'd have a hard time trusting an LLM to do this correctly. My standard is accurate semantics lifting from Machine Code to C. Reversing assembly to C is very delicate. There are many patterns that tend to usually map to obvious C constructs... except when they don't. And that assumes the original source was C. Once you bump into routines that were hand-coded assembly and break every established rule in the calling conventions, all bets are off. I'm somewhat convinced that decompilation cannot be made fully-automatic. Instead a good decompiler is just a lever-arm on the manual work a reverser would otherwise be doing. Corollary: I'm also somewhat convinced that only the decompiler's developers can really use it most effectively because they know where the "bodies are buried" and where different heuristics and assumptions were made. Decompilers are compilers with all the usual engineering challenges, plus a hard inference problem tacked on top.

All that said, I'm not a pessimist on this idea. I think it has pretty great promise as a technique for general reversing security analysis where the reversing is done mostly for "discovery" and "understanding" rather than for perfect semantic lifting to a high-level language. In that world, you can afford to develop "hypotheses" and then drill down to validate if you think you've discovered something big.

Compiling and testing the resulting decompilation is a great idea. I do that as well. The limitation here is TEST SUITE. Some random binary doesn't typically come with a high-coverage test suite, so you have to develop your own acceptance criterion as you go along. In other words: write tests for a function whose computation you don't understand (ha). I suppose a form of static-analysis / symbolic-computation might be handy here (I haven't explored that). Here you're also beset with challenges of specifying which machine state changes are important and which are superfluous (e.g. is it okay if the x86 FLAGS register isn't modified in the decompiled version, probably yes, but sometimes no).

In my case I don't have access to the original compiler and even if I did, I'm not sure I could convince it to reproduce the same code. Maybe this is more feasible for more modern binaries where you can assume GCC, Clang, MSVC, or ICC.

At any rate: crazy hard, crazy fun problem. I'm sure LLMs have a role somewhere, but I'm not sure exactly where: the future will tell. My guess is some kind of "copilot" / "assistant" type role rather than directly making the decisions.

(If this is your kind of thing... I'll be writing more about it on my blog soonish...)