Comment by aengelke | Hacker News Reader

Comment by aengelke | Hacker News Reader

Parent Full thread

> Rolling your own infrastructure is not a great idea IMO

For the start, I'd agree, but as soon as you want to do non-trivial things, I don't think that standard compiler IRs (e.g., LLVM) are a good way for binary analysis. (POV: been there, done that.) The time investment in fighting the existing IR quickly becomes larger than just writing a custom IR, which is not _that_ much effort. Running O2 optimizations sounds great in theory, but is often not too helpful in practice.

LLVM is clearly intended to provide a (not too large) set of semantics that can be mapped easily to different architectures. This is great for compilation, but not for the other direction, as even simple things like add-with-carry end up being multiple non-trivial instructions which aren't handled by existing optimization patterns. The same goes for vector instructions, where many patterns are encoded in the back-ends and not in the IR passes. So you end up writing custom passes fairly quickly, but still fight against an IR that doesn't allow you to express the things you'd like to have.

In the long run, a custom IR allows for a much more idiomatic code representation and more flexible analyses; not to mention that LLVM-IR is fairly heavy-weight and not particularly efficient. If I were to write another binary translator (I wrote Instrew/Rellume based on LLVM), I would definitely not choose LLVM.

I completely agree on the SMT part. I experimented with SMT, but standard data flow analysis suffice 90% of the time and are much more efficient and predictable.

> Running O2 optimizations sounds great in theory, but is often not too helpful in practice

I cannot stress how important it is to have at your disposal an alias analysis framework and analyses such as LazyValueInfo and ScalarEvolution. In fact, some whole key features of rev.ng are possible thanks to them. Either one (thinks) they're not need them or you have to reinvent the wheel. People think binary analysis is something special, it's not, just compilers. And there's really really a lot to learn from the design of LLVM.

Add with carry doesn't sound like a gigantic problem to me, given what you get in return. About vector instructions, a lot of autovectorization takes place in the mid-end and LLVM supports vector types, but I'm not sure if you're talking about them in input or in output.

If you really want to roll your own thing, one could at least use MLIR, where you can define your own operations but you reuse MLIR infrastructure, pass manager, DCE and much more. However, you'd lose the above mentioned passes in the early stages of the pipeline is too much of a loss.

As mentioned, we started facing the limitations of LLVM when you want sophisticated types (e.g., unions) and more and more high level concepts while you get close to C. That's why we're rolling our MLIR dialect, clift.

> LLVM-IR is fairly heavy-weight and not particularly efficient

I guess it depends what's the comparison, but in our experience, it scales pretty pretty well.

But overall... no one wants to learn your custom IR, there's much more incentive in learning an (the most?) established IR, in particular for a lifter. If you use LLVM IR you get AddressSanitizer, CoverageSanitizer, KLEE, polly, libFuzzer, bindings, documentation and so much more.

> People think binary analysis is something special, it's not, just compilers.

I (conceptually) agree! But there's different sorts of compilers, and they have different requirements, which (should) materialize in the code representation. An offline C/C++ compiler has different requirements than a JIT JavaScript compiler, and so their architecture is very different. The IR should be express the relevant operations/data structures well and should easily support the analyses/transformations. If LLVM is a good fit for you, that's great. But in many cases, it's not.

I've been bitten from being warned about not writing my own IR (effort concerns), and in hindsight, it would've been the much easier way, both for low-level and high-level transformations. IMHO, people should not be afraid to roll their own IR and make it a good fit for their use case.

> one could at least use MLIR

MLIR is a nice idea (and great for selling/publishing). I find the implementation to be somewhat lacking and constraining, difficult to work with, and not particularly efficient.

> no one wants to learn your custom IR

That's a problem, but there's no silver bullet.

I see your position, but I've to say that in 8 years of rev.ng we revised many many design choices we initially made, but not using LLVM was not one of them, so I can't agree.

Rolling your own IR is not hard, in fact, it's very easy (and sometimes we roll temporary IRs for specific purposes), but it's also true that most IRs look more or less the same. QEMU's tiny code and LLVM IR, for instance, are quite similar. What matters is the ecosystem around them, and the ecosystem around LLVM is really great.

Anyway, every Friday morning 11:00 CEST we have the rev.ng hour, our weekly internal technical meeting. It would be interesting to discuss what limitations you hit with LLVM that we did not hit and your concerns about efficiency of LLVM and MLIR.

We used to stream the rev.ng hour and we're in the process of going back to publish some talks every now and then. If you're interested, drop me an e-mail! :)

Reply on news.ycombinator.com