PeachPy: Assembly Code Generation in High-Level Python | Hacker News Reader

PeachPy: Assembly Code Generation in High-Level Python

github.com

116 pointsshagunsodhani29 comments

I remember coming across ARM's VIXL (https://github.com/armvixl/vixl), which is a similar concept but for C++, and just being blown away by how easy it was to twiddle the stack and assemble new functions on the fly - yee haw!

Cool! I have been doing this kind of programming with dynasm [*] and finding it addictive. I wonder if assembler is going to make a come back :).

https://github.com/luapower/dynasm/blob/master/dynasm.md

I wonder if assembler is going to make a come back :).

I'm certainly seeing a lot of cool projects that make assembly more accessible, not to mention a lot of good guides on the web. As for myself, I've recently made Python ctypes bindings for GNU Lightning, which lets you write architecture agnostic RISC assembly, then JIT to native code on x86, ARM, etc. See https://github.com/cslarsen/lyn/

It's pretty light-weight and JITs fast. The point is that it's very fun and addictive to do assembly from a language like Python.

I proposed a while back to replace inline assembler with inline LLVM bytecode. Do the portable code thing in general except accelerate the algorithms with the bytecode. Toolchain produces assembly from that. Probably still have real assembly where it's needed for hardware manipulation, etc but I'd think you'd get quite the performance/optimization improvement with that over C.

What you think about LLVM as cross-platform assembly?

It's pretty light-weight and JITs fast. The point is that it's very fun and addictive to do assembly from a language like Python.

Because you skip the boring boilerplate, and go straight to the fun parts!

My biggest problem with assembly was coming up with names for all of the labels :)

One of the hardest problems of CS... naming!

How does PeachPY handle loops? I'm looking for examples, but will it generate a label for me and then let me specify the contents only?

Edit: https://github.com/Maratyszcza/PeachPy/blob/master/examples/... !

(Add a bit more, give it some static typing and soon we'll be coding C in Python)

That's how you unroll a loop, not how you write one. As far as I can tell in PeachPy you can do something similar with anonymous labels:

  counter = GeneralPurposeRegister64()
  MOV(counter, 64)
  with Loop() as loop:
    ...
    DEC(counter)
    JNZ(loop.begin)

Thanks for the clarification, that's still a decent improvement.

It likely won't for mainstream but it's always been popular in niche areas. Here's a few projects using it and pages on it:

KolibriOS http://kolibrios.org/en/

Anything by 2ton and w/ their library https://2ton.com.au/Products/

Gibson's guide on Win32 Assembly w/ links https://www.grc.com/smgassembly.htm

Hyde's AoA and High-level Assembly are cool http://www.plantation-productions.com/Webster/

Or for the elite: 8-bit GUI desktop http://www.symbos.de/

There was a bit of hate for Hyde's HLA last time I posted a link. I'm really not sure why, theoretically it's a great intermediate between C and ASM.

Same thing happened to me when I posted. The gist of it is they thought you weren't learning real ASM thinking that way. I thought the point was to temporarily ignore more difficult parts of ASM to make learning incremental and easier. And then be able to do same as real assembly coder later w/ ability to make tradeoffs between raw performance, productivity, and easy-to-read code. Overall, a great tool in theory.

Also, gave me ideas on how to bootstrap a subversion-resistant system. You start with assembler, build a HLA-like tool from that, easily build a 3GL + straight-to-HLA compiler with that, and then use that to build rest (including optimizations). My concept was inspired by Wirth and Jurg's bitslice -> M-code ASM -> Modula 2 -> OS work in Lilith project. Alternatively, you can go MenuetOS or KolibriOS w/ HLA for above benefits. Use full HLA for rapid addition/testing of new features with raw ASM for optimizing the fast paths. Thoughts on either?

A typed assembly would be in order.

Go get, use and improve them:

http://www.cs.cornell.edu/talc/

http://research.microsoft.com/en-us/um/people/nick/coqasm.pd...

> I wonder if assembler is going to make a come back :)

Are you trolling? Assembler hasn't gone anywhere nor will it ever.

Great tool. It supports assembly generation for the golang Plan 9 toolchain as well. The author, Marat Dukhan, gave a great talk on this last night at the Atlanta Go Meetup. Slides are at https://docs.google.com/presentation/d/1MYg8PyhEf0oIvZ9YU2pa...

Debugging would be a nightmare when you have 300+ line function. https://github.com/Maratyszcza/PeachPy/blob/master/codegen/x...

This is pretty cool stuff. You get the RAD advantage of Python, probably easy to extend w/ new features, customization of real assembly, and then the assembling part itself. Nice tool. Assembly hasn't been this interesting since Microsoft started coding it in a proof assistant [1]. I like this better as Python is a bit more accessible. ;)

[1] http://research.microsoft.com/en-us/um/people/nick/coqasm.pd...

I've been looking for a way to assemble ARM instruction by instruction in python; unfortunately this only seems to do x86.

A brief look at the source reveals ARM support also.

This is a re-write of an older version of PeachPy (you will find it at https://bitbucket.org/MDukhan/peachpy), which supported x86-64 and ARM, but didn't generate object files directly. The new PeachPy currently only supports x86-64, albeit there are remnants of old code for ARM.

I didn't get the example in section "PeachPy as Instruction Encoder" to work for arm on the linked github version, the version currently on pypi, or the older bitbucket version. If the module loads at all, it seems to always complain about lacking function context.

Probably I didn't make it clear enough:

- The old version of PeachPy (on BitBucket and PyPI) supports ARM, but can't directly generate machine code.

- The new version of PeachPy (on GitHub) can directly encode instructions into machine code, but currently only supports x86-64. The files in peachpy.arm module are merely non-working remnants of the old code.

In the early-mid 1980s there were several projects claiming this high level language to Assembly Code or P-Code (Machine not Microsoft's abandoned p-code). They never really worked out right and due to weird optimizations could be just as fast or 5 times slower.

How is an assembly code generator different than a compiler?

Earlier versions of PeachPy generated assembly source that had to be compiled with NASM or GNU assembler. Now PeachPy can directly generate object files, and it would be fair to call it a compiler, but I like the old name

Assembly code then has to be compiled into machine code to create the executable.

To expand a bit in terrible pseudocode that I sort of remember from Atmel ucs:

Assembly: mov 0x1,reg2 # put 1 in register 2

Machine Code: 0xAC12 # Move an immediate value into register 2

Assembly: mov *reg2,reg1 # put contents of address in reg2 in reg1

Machine Code: 0xBF201 # Move an indirect value, offset 0, address in reg2 to reg1

It's a much more straightforward transformation than the one from C->ASM though.

Edit: This link might make more sense... the difference is mainly between the mnemonic and the encoding: http://www.mrc.uidaho.edu/mrc/people/jff/digital/MIPSir.html

You'll also notice there are explicit instructions for immediate ops, but no two assembly schemes seem to share the same level of abstraction.

Url changed from http://lambda-the-ultimate.org/node/5251, which points to this.

Thanks!

Reply on news.ycombinator.com