x64asm: C++ Library with in-memory assembler, parser, and linker | Hacker News Reader

x64asm: C++ Library with in-memory assembler, parser, and linker

github.com

89 pointstomcam20 comments

How do W^X restrictions affect something like this? I assume there must be a way to work around it given the prolificacy of JITing VMs, but interested in the details. The last time I was fiddling with machine code in memory was before W^X was a thing.

Use mprotect to change the access protection flags on a page of memory. Assuming it is initially allocated as write but no execute (PROT_READ|PROT_WRITE), write your machine code to it, then change it to read-only and execute (PROT_READ|PROT_EXEC).

http://man7.org/linux/man-pages/man2/mprotect.2.html

mprotect is part of POSIX, so works on Linux, macOS, etc.

The Windows equivalent is VirtualProtect

https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...

Annoyingly this doesn't apply on the most modern platforms... see windows phone/universal etc as well as mobile platforms, consoles and all sorts of embedded systems.

(ps I did a significant chunk of this for https://code.google.com/archive/p/fridgescript/ back when this made real sense, I'm not just making hot air... this is quite a mediocre result imo ... bedroom stuff from years ago..,)

There are AsApp variants on UWP, if you claim the JIT permission.

I've written a demonstration of this sort of thing in Ruby, where I scribble a cdecl function to memory and execute it, with the end goal being: invoke the cpuid instruction to get info about the processor.

http://www.cstrahan.com/posts/2013-07-15-pure-ruby-cpuid-via...

I must be very dense today - what exactly is this? I'm a tad confused...

Seems to be a slightly over-designed (amd64) assembler in the form of a library, plus some (in-library) tools that are slightly cool but nobody will use.

> tools that are slightly cool but nobody will use.

It seems like this project could be quite useful for learning, though.

fuuuuh, I could have used this 2.5 years ago when I was messing around with a JIT compiler for elementwise array operations. Started with a VM, wanted to move to x86 but gave up due to difficulty of codegen.

Why not use LLVM?

I have tried using LLVM as a JIT for Lua [0] (7 years ago), but found the codegen to be too slow [1] and use too much memory. It might have improved since then, but I really don't think it can compete with LuaJit's highly optimized tracing JIT.

Others [2] have also found LLVM to be unsuitable as a JIT.

0. https://github.com/Neopallium/llvm-lua

1. http://stackoverflow.com/questions/4077396/llvm-jit-speed-up...

2. http://stackoverflow.com/a/6833494/98107

Is he trying to solve jit though? Maybe he is sick of native full compile times... e.g. He might be working on a jit :I

I followed reference [2] and the highest voted answer (not the selected one) concludes:

> I think the results speak for themselves and demonstrate unequivocally that LLVM is perfectly suitable for JIT compilation.

I don't think that answer was there when I had first come across that question (back in 2010-2011).

That answer is about typed languages. The issue is with dealing with dynamic types. There is a lot more code generated for the JIT to compile.

Why not use DynASM, especially since you are using the Lua eco-system already?

https://luajit.org/dynasm.html

DynASM isn't that portable... not remotely even near LLVM

LLVM-lua was just an experiment. I wanted to see what kind of performance statically compiled Lua could achieve.

I didn't really need the JIT feature. But I did add support for it because Lua code can dynamically generate/load more Lua code.

I just found that the runtime overhead (memory and stack space) of LLVM was too high for my needs.

Have you had a look at the ORC JIT API?

I hadn't heard about that. I have been working on other projects, so haven't had the time to keep up with the developments in LLVM.

Thanks for the heads up.

The main issues I had with LLVM was the partly do to the memory usage. It can product very good machine code, but the time needed to JIT a block of code was sometimes longer then just running the it in the interpreter.

One of the features I like about Lua the most is the low resource overhead. LLVM was just too big for what I was looking for.

Usage LLVM to JIT/AOT compile Lua code was really just an experiment, something I wanted to try out.

I re-wrote [0] the llvm-lua project with a simple C code-gen backend for compiling Lua code to native code. Which is a bit more useful, since it is easier to cross-compile pure C code, then LLVM bitcode. A lot of the people that tried using llvm-lua wanted to cross-compile Lua code and embed it into an iOS application.

0. https://github.com/Neopallium/slua

x10^999 +1s

Reply on news.ycombinator.com