CompilerGym: A toolkit for reinforcement learning for compiler optimization | Hacker News Reader

CompilerGym: A toolkit for reinforcement learning for compiler optimization

facebookresearch.github.io

139 pointsazhenley16 comments

I've been helping on the project, it's lead by Chris Cummins and Hugh Leather.

Just a heads-up for folks, we haven't fully cleaned up and gotten ready for public attention yet, we are 90% there.

Once we are golden, we're going to write a note with a way to submit the results of your own agents, compare with baselines (random, actor-critic), etc.

And as others noticed, in it's current form we're focusing on code size and phase ordering, but we will be expanding over time to other optimization problems like runtime.

This is really cool - I understand how the reinforcement loop works for improving performance, but how does it verify that the optimizations applied don't change the semantics/correctness of the code?

Regular old tests, I imagine

This. For now we rely on differential testing against a gold-standard implementation (e.g. unoptimized). For the action space we expose, any semantics-breaking change induced by our tool is a compiler bug.

Holy smokes, this is so fucking cool.

ML is neat conceptually, but as far as practical applications this really excites me.

Where do they benchmark code candidates? This is hard, noisy and expensive (I see example reward for code size).

I am not an expert, but bayesian learning maybe more appropriate for such an expensive-sampling environment?

They are optimizing only for code size, so they don't need to run the code: they only count bytes emitted.

This works by changing around the order / interleaving of various LLVM optimization phases, so the learning process does not require knowledge of program timing or correctness.

I heard when they started to do this, LLVM folks were freaked out because they worried changing orders may affect correctness. But it seems to work so far :)

I dunno. People get excited about this, but compiler optimizations for C just don't do that much, and I haven't seen a lot of useful results coming from searches in the optimization parameter space. Optimizations with magic number parameters are usually poorly thought out or attempting to cap cpu/memory usage.

It's more useful for C++ because inlining actually can benefit from this, as well as some other passes with speed/code size tradeoffs. So it might work here.

What a great idea. Would per-instruction/opcode costs make this easier to optimize?

Are MuZero or the OpenAI baselines useful for RL compiler optimization?

https://github.com/werner-duvaud/muzero-general

https://github.com/openai/baselines

This looks pretty cool. FB does crank out some cool toolchain stuff.

ML cannot solve the halting problem.

But it can optimize heuristics to death.

How does pass reordering relate to the halting problem?

You are right. Looks like it is just trying different permutations of the compiler flags, not actually trying to generate optimization proofs using machine learning. The latter is what my comment is aimed at.

Compression can't solve the pigeonhole problem either. And yet we're using it right now.

Most of the interesting stuff I want to accomplish is avoiding intractable problems, if for no other reason than that our peers will accuse us of tilting at windmills or boiling the ocean.

Finding the problem you can solve is effectively relaxation.

Reply on news.ycombinator.com