Comment by tralarpa | Hacker News Reader

Comment by tralarpa | Hacker News Reader

Parent Full thread

> It will be years before Ghidra

Can you elaborate? I would really like to see what the HexRay decompiler does better (or worse) than Ghidra, but I am too poor to buy it (and dogbolt.org is not interactive, so I cannot edit function signatures to "help" the decompiler etc.).

Is it better in general, for a specific programming language or platform (e.g. C++, Windows), or for a specific use case (e.g. obfuscated code)? I heard about their cloud-based stuff, although I don't know what they are exactly doing there. Maybe ML trained on source code? Function signatures of the latest malware?

After several hundred hours with Ghidra, I think it certainly would need some polishing, in particular:

  - UI. Too many frequently-used dialogs are not optimized for keyboard usage.
  - Decompiler too stubborn sometimes, ignoring user input (e.g. manually specified types).
  - Decompiler needs better heuristics for the treatment of some common cases (e.g., often doesn't recognize for-loops and array accesses)
  - Quite dangerous: Sometimes the decompiler gets lost, especially if a function contains handwritten assembly code with unusual control flows. Okay, can happen. But instead of displaying a warning it just shows you the part it could decompile and you have to figure out by yourself that something is missing.

But most of the above issues are fixable. Instead, I would be interested in learning about more fundamental differences between the two decompilers.

The IDA decompiler is just generally more complete and more polished than Ghidra for the architectures it targets.

One issue with Ghidra's that I keep hitting is its poor support for amd64 SIMD. There's a good example at <https://github.com/NationalSecurityAgency/ghidra/issues/249>:

  0000000000000000 <intrinsics>:
     0: f3 0f 1e fa           endbr64 
     4: 0f c6 ca 1b           shufps $0x1b,%xmm2,%xmm1
     8: 0f 58 c1              addps  %xmm1,%xmm0
     b: c3                    retq

IDA produces great decompilation:

  __m128 __fastcall intrinsics(__m128 a1, __m128 a2, __m128 a3) {
    return _mm_add_ps(a1, _mm_shuffle_ps(a2, a3, 27));
  }

The Ghidra output is a mishmash of CONCAT pseudo-macros.

> - Decompiler too stubborn sometimes, ignoring user input (e.g. manually specified types).

This is the thing that sticks out the most IMO. IDA decompiler is quite a bit more flexible than Ghidra's. When you assert a type, it will usually not ignore it. It may sometime get a bit lost if you give conflicting types to dependant variables, but otherwise, it's pretty good at this.

One of the annoying bits of ghidra (though it may have improved, it's been around a year since I last used it) is that there's no way to "split" a variable. Sometimes, ghidra will have some code that looks roughly like:

    int x;

    x = 0;
    doSomething(x);
    x = 1;
    doSomething2(x);

(Obviously, really simplified).

The problem is, sometimes, x needs to be an int for the first function, and a bitflag structure for the second. But Ghidra has no way to say, "hey, from this assignment on, treat `x` as another variable", so you have to either generate a union (ugly) or deal with sending the wrong type (also ugly).

IDA tends to be much better at this. All variables start out as "split" as it can make it (almost in static single assignment form). Then, the user can tell it "Those two variables are actually the same, please treat them as one". I find this flow works really, really well.

In my experience, in Ghidra you can Ctrl+L to mark a variable as a different type, but retyping or renaming sometimes fails because it creates a HASH variable which sometimes fails to apply to the decompiler output (instead creating a new variable of the right type but the code still references the old wrong one).

Also can I haz offset pointers pretty please? It's not in the last released version I tried, and the last Git version I tried had offset pointers but they didn't affect decompiler output so multiple inheritance and container_of linked lists still came out broken.

> is that there's no way to "split" a variable

Right click -> “Split Out As New Variable”, but it seems like this doesn't work for stack reuse yet (just registers, or more generally, simple varnodes).

https://github.com/NationalSecurityAgency/ghidra/issues/2573

This was my biggest frustration when first using Ghidra (granted, I didn't spend much time reading the documentation before diving in). Big thanks to the people responding to your comment with workarounds!

Sure, the thing about decompilation is it's by nature taking lossy data and trying to turn it into something that is almost lossless. The working theory of most decompilers, including Ghidra's, is to have a base level of "translation" ability between asm and C (for example). However, there's a ton of stuff that is missed in this base level. Every compiler speaks it's own dialect of assembly, and different compilers prefer different optimizations, code removal, etc as a strategy. As a result a good decompiler also includes heuristics that are expensive to create and learn about. Hex Rays has dedicated an entire company to finding these heuristics in many compilers across many platforms. This isn't knowledge that is easy to acquire or maintain. I think that Ghidra has the right sauce to do it (a large community) but it will be a while before that community can organically produce the same results.

Due to these heuristics IDA produces real actionable code quicker. My experience with Ghidra (less than yours) is that it produces mostly garbage on a lot of different things and it requires a lot of prep work to make truly usable. This might not be noticeable on small or simple binaries but on larger binaries it actually becomes a real measurable problem. While Hex Rays isn't perfect, it's about as close as we can come to it right now and it generates very human-looking code with smart optimization removal. One thing I remember with Ghidra not long ago was a common optimization like using SSE registers for arrays would produce a page worth of non-sense for something simple. Additionally, detection of standard libraries still isn't good so you end up wasting your reversing time on re-reversing a different compilers version of strlen than actually doing the work you need to do. If you could use FLIRT signatures in Ghidra legally I'd imagine Ghidra would be vastly improved.

My PhD work is about detecting standard library functions in obfuscated/stripped binaries. My most recent model gets about 70% accuracy for a handful of musl-linux-x64 functions. No signatures, just machine learning and disassembly from r2.

I'm not a reverse engineer, but I have been working on this with the assumption that it would be a good time-saver for a reverse engineer. Feel free to DM me if you'd like to discuss.

OTOH, I was able to add decompiler support for a new architecture (or1k) to Ghidra in two weekends just by following examples and without pre-existent Ghidra internals knowlege and without writing any Java code. To me that's an awesome flexibility.

Reply on news.ycombinator.com