Hacker News Reader

Comment by wingo | Hacker News Reader

AFAIUI libc is in the initrd, but it's a statically linked libc. Guile's dynamic FFI uses dlopen/dlsym to get its function pointers, so it can't do that in a statically linked libc. (Or could it? I suppose there's no reason why a statically linked library wouldn't expose symbol tables, whether via dlsym or some other means.)

Your point about size is well-taken though. Besides libc in this case, Guile's build products are much larger (and slower) than LuaJIT's images because we've been trying to write the compiler in Scheme from the very beginning. We can't even think about doing a nice JIT until we have a good AOT compiler, because otherwise the JIT written in Scheme would be too slow.

View on HN

I believe that LuaJIT's ffi can use symbols that are statically linked in, but I haven't tested it myself so I could be wrong. You can't use dlsym though in a static binary, so it might need some work. My plan to port ljsyscall to straight Lua without ffi though is to generate C bindings rather than writing them by hand. But I have done all the ioctl stuff in Lua not C so all I need is all the syscalls and structs defined and then all the ffi code for ioctls, netlink etc will just work; all the constants and so on are define on the Lua side (that has other issues like dealing with the ABI directly in Lua).

I might be tempted in your case just to use a huge initrd for now, it will be freed after boot anyway.

LuaJIT's "running" page http://luajit.org/running.html suggests linking with -E (gcc -Wl,-E) to make all your global symbols visible to dlsym() even when you statically link.

That's what we're doing in Snabb Switch http://snabb.co/snabbswitch/ to create a statically linked executable that heavily uses FFI. (Currently don't statically link libc though so I don't know if there is a particular gotcha there.)

I think it won't work if you statically link LuaJIT too. I think it still uses dlopen, which with glibc still needs the dynamic libc available at run time, and might not work with other libcs. But you could fake it all with a statically linked table of function pointers quite easily I guess.

Modulo the bug at http://sourceware.org/bugzilla/show_bug.cgi?id=15022 it's possible to dlopen symbols from the static binary.

However, all libc symbols not used by Guile are omitted from the resulting static binary, which is why the FFI is not an option. (This is independent of Guile or the libc brand.)

As for the size, the initrd is less than 5 MiB (the 'guile' binary is less than 4 MiB).

If you dynamically linked it would be quite a bit bigger, libc alone (none of the other libs like libm libdl etc) is 1.8MB on my system. But it should be manageable. Or you can force the symbols to be linked in by adding a structure with the symbols in.

Reply on news.ycombinator.com