Avo: Better X86 Assembly Generation from Go | Hacker News Reader

Avo: Better X86 Assembly Generation from Go

github.com

77 pointsingve19 comments

I really wish Go (really, Plan 9) hadn't made their own assembly syntax. The fact that we have AT&T syntax to deal with instead of just using Intel's syntax is bad enough.

I understand the rationale—that you can use the assembly syntax as a sort of IR by unifying the syntax of certain basic operations across different ISAs—but that's properly considered a low-level IR, not assembly. (It's also a pretty bad form of IR for doing any interesting optimizations, which is why Go uses a separate SSA form for that.)

Author of avo here. While I did really enjoy working on this project, I still fully agree with you.

Go (Plan 9) syntax is not sufficiently different to make it a powerful intermediate representation, hence SSA as you point out. In fact many of the differences are frustrating subtle changes in instruction mnemonics or operand order. This might be okay if there was a canonical instruction database that made all these differences from Intel and AT&T easy to discover. However these exceptions are really recorded in various parts of the golang.org/x/arch/x86 repo. Producing the instruction database for avo was quite a time-consuming process. See:

https://github.com/mmcloughlin/avo/blob/master/internal/load... https://github.com/mmcloughlin/avo/issues/23

I've gained a lot more familiarity with the assembler and supporting infrastructure, so I'm hoping I may be able to contribute to this area of the core Go project.

Yeah, AT&T syntax has much of the same issues of poor documentation. In particular, operand order is easy to remember for 2-operand instructions: it's reversed in AT&T. But now with SSE and AVX we have 3- or 4-operand instructions, and the AT&T versions sometimes put the operands in (2, 1, 3), sometimes in (3, 2, 1), etc. There's no documentation for this aside from GCC as far as I can tell. :(

Yes! This tripped me up at one point. My comment says "Otherwise we could be in the bizarre special-case of 3-argument CMP instructions."

https://github.com/mmcloughlin/avo/blob/0e253b3753beb409b3dd...

It gets to the point where the only source of truth is the Go assembler itself. My test suite takes the avo instruction database and generates a massive assembly file with one line per instruction form, then passes this to `go tool asm`.

Even just mapping between the mnemonics is non-trivial. You want to think that it's just a matter of removing size suffixes from AT&T, but it's sadly not that simple. "movslq" in AT&T maps not to "movsl" or even "movs" in Intel, but "movsxd". How is anybody supposed to figure that out?

I've done a reasonable amount of ARM, x86 and amd64 programming with the go assembler.

It's quirks are annoying to start with, but if you use more than one assembly language you begin to see the advantages of a more consistent syntax between them.

I suspect the advantages for the go maintainers of a uniformish assembler syntax are the real reason for it.

I use ARM and x86-64 extensively and any minor inconvenience caused by different syntax for trivial operations is completely outweighed by the downside of being incompatible with the CPU vendor's manual.

LLVM MachineInstr and Cranelift do this much better than Go does, by maintaining a clean separation between the low-level IR and the assembly language.

This is a nice project :-)

Any chance of ARM/ARM64 support?

It would be super useful if you could generate the code to memory then execute it directly.

I believe this assembly is golangs assembly, which is ISA neutral

Only a small number of instructions in Go assembly are architecture neutral.

avo currently implements the x86 subset of Go assembler. It's definitely possible to add support for additional architectures, but it may be a fairly significant project.

Thanks :)

> Any chance of ARM/ARM64 support?

Yes multi-architecture support is definitely something I've had in mind. However I wanted to limit the scope for the initial version, so I could produce a result in a reasonable time and allow me to gauge interest.

> It would be super useful if you could generate the code to memory then execute it directly.

avo was inspired by PeachPy and asmjit, both of which can produce executable code directly. So implementing an assembler in avo is something I have had in the back of my mind. However it wasn't one of my primary goals.

The use case I initially had in mind is massively unrolled crypto routines, where there's a lot of assembly code required but not much going on conceptually. My AES-CTR mode CL (https://go-review.googlesource.com/c/go/+/136896) and meow hash port (https://github.com/mmcloughlin/meow) got me thinking in this direction.

A similar concept for python: https://github.com/Maratyszcza/PeachPy

avo author here. Yes PeachPy was the inspiration for this, as well as asmjit. Both mentioned in the credits:

https://github.com/mmcloughlin/avo#credits

PeachPy actually has Go output already, and Damian Gryski has used this to great effect:

https://github.com/mmcloughlin/avo/issues/40

Unfortunately because PeachPy wasn't initially written with Go in mind there are some rough edges and Go features it simply doesn't support. avo hopes to fill this gap for Go programmers.

Ah - peach -> avocado, I see that now.

Haha yes! PeachPy was written at Georgia Tech and avo was written in California.

We actually had Marat come to the Go meetup in Atlanta when he was finishing up his PHD at GT. He had done some preliminary work on SIMD. It was a small meetup ( I think only ~6-7 in attendance) and went over many folks heads, since a lot of people were focused on breaking up Ruby monoliths into Go. SIMD would have been nice, but really we were still reveling in the benefits of getting the hell out of ruby for high-throughput networked services. https://docs.google.com/presentation/d/1MYg8PyhEf0oIvZ9YU2pa...

Oh cool, thanks for sharing that talk. I might try to submit an avo talk to a meetup, if anyone will have me :)

I love this

Really dope - excited to see this project come out!

Reply on news.ycombinator.com