Hacker News Reader

Comment by lol768 | Hacker News Reader

    $ chmod +x llamafile-server-0.1-llava-v1.5-7b-q4
    $ ./llamafile-server-0.1-llava-v1.5-7b-q4 
    run-detectors: unable to find an interpreter for ./llamafile-server-0.1-llava-v1.5-7b-q4

Hmm. Did I do something wrong? (Ubuntu 22.04 / )

Installing the portable binfmt_misc gets me further, but still:

    $ ./llamafile-server-0.1-llava-v1.5-7b-q4 
    zsh: permission denied: ./llamafile-server-0.1-llava-v1.5-7b-q4

    $ sh -c ./llamafile-server-0.1-llava-v1.5-7b-q4
    sh: 1: ./llamafile-server-0.1-llava-v1.5-7b-q4: Permission denied

View on HN

You can solve the run-detectors issue with:

    sudo wget -O /usr/bin/ape https://cosmo.zip/pub/cosmos/bin/ape-$(uname -m).elf
    sudo sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"
    sudo sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register"

You can solve the zsh permission denied issue by either (1) upgrade to zsh 5.9+ (I upstreamed a fix for this bug in zsh two years ago) or (2) use the sh -c workaround you discovered. If that one doesn't work, then it likely needs to be chmod +x. If the execute bit is set, and your sh still isn't working, then please let me know, because I'm not aware of any sh that still doesn't support APE.

See the Gotchas section of the README https://github.com/mozilla-Ocho/llamafile#gotchas

That worked, thanks Justine! I use fish, so I didn't get a zsh error, but I had missed the Gotchas section (and the README), so this helps!

Fish is another cool shell I got to help improve two years ago by upstreaming a patch for this. So long as you're using a recent version, you should be golden (provided binfmt_misc doesn't cause any issues). Let us know what you think of llamafile!

Thank you, I really like it! It's a very clever way to get LLMs deployed, and with Cosmopolitan, I don't need to point people to different downloads for the same LLM. Excellent job.

Thanks!

Yet another jart tour-de-force. I knew I had to sponsor you on Github back when I read your magnificent technical breakdown of APE, lol.

(sorry for OT!)

You're awesome!

Last thing you need is to chmod +x the interpreter: chmod +x /usr/bin/ape (it is indeed not in the README)

This was it, wget creates the file and it's non-executable (and I'd already double checked the actual llamafile was executable, but had missed this) - thanks!

I get the same error, and there's no `ape` file to make excecutable, hm.

You can manually download the `ape` command from https://cosmo.zip/pub/cosmos/bin/ Please see the Gotchas section of the README for the copy/pastable commands you can run: https://github.com/mozilla-Ocho/llamafile#gotchas

Reply on news.ycombinator.com