Comment by jart | Hacker News Reader

Comment by jart | Hacker News Reader

Parent Full thread

llamafile runs on all Linux distros since ~2009. It doesn't have any dependencies. It'd probably even run as the init process too (if you assimilate it). The only thing it needs is the Linux 2.6.18+ kernel application binary interface. If you have an SELinux policy, then you may need to tune things, and on some distros you might have to install APE Loader for binfmt_misc, but that's about it. See the Gotchas in the README. Also goes without saying that llamafile runs on WIN32 too, if that's the world you're most comfortable with. It even runs on BSD distros and MacOS. All in a single file.

It doesn't seem to run on NixOS, though I'm new to Nix and may be missing something.

    $ ./llava-v1.5-7b-q4-server.llamafile --help
    ./llava-v1.5-7b-q4-server.llamafile: line 60: /bin/mkdir: No such file or directory

Regardless, this (and Cosmopolitan) are amazing work - thank you!

The APE shell script needs to run /bin/mkdir in order to map the embedded ELF executable in memory. It should be possible for you to work around this on Linux by installing our binfmt_misc interpreter:

    sudo wget -O /usr/bin/ape https://cosmo.zip/pub/cosmos/bin/ape-$(uname -m).elf
    sudo sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"
    sudo sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"

That way the only file you'll need to whitelist with Nix is /usr/bin/ape. You could also try just vendoring the 8kb ape executable in your Nix project, and simply executing `./ape ./llamafile`.

`./ape ./llamafile` worked immediately and without problems I can see - thank you!

Reply on news.ycombinator.com