Hacker News Reader

Here's a bit of a clue to how it works:

    ./python 
    Python 3.11.4 (heads/pypack1:65ac8ac, Oct 16 2023, 02:35:05) [GCC 11.2.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import sqlite3
    >>> sqlite3
    <module 'sqlite3' from '/zip/Lib/sqlite3/__init__.pyc'>
    >>> import pathlib
    >>> list(pathlib.Path("/zip").glob("*"))
    [PosixPath('/zip/.symtab.amd64'), PosixPath('/zip/.symtab.arm64'), PosixPath('/zip/usr'), PosixPath('/zip/.cosmo'), PosixPath('/zip/Lib'), PosixPath('/zip/build')]

From Python's point of view there a "/zip/" folder with stuff in it, presumably stuff that's hidden inside the fat binary.

Yup, this is basically it. If you check out the "Actually Portable Executable" blogpost by the same author [1] and go to the "PKZIP Executables Make Pretty Good Containers" section there's a note on the approach.

1: https://justine.lol/ape.html

absolutely impressive

The "/zip/" folder refers to the ZIP store inside the actually portable executable -- you can run `unzip -vl ./python` to see what's inside.

Also, to add new pure-python packages, you just need to unzip them into a local folder "./Lib", and add that folder to the APE.

It’s so nice seeing portability over binary size winning all around these days. Such a breath of fresh air. There might be legit cases but for some reason so many people are still hang up on ´but binary size’ for no logical reason whatsoever.

I think a lot of the "logical reason" is driven by more the ephemeral, sandboxed, execution tools. e.g. a `docker run` or `serverless` architectures, where full root filesystems are populated on demand. Less executable data size there can go a long way in reducing latency/price.

> Less executable data size there can go a long way in reducing latency/price.

According to the article, Cosmo's fat binary dash was 30% larger but ran faster and used less real memory than Alpine's x86-linux-only version.

However there is still probably a 30% overhead for storage and copying.

> It’s so nice seeing portability over binary size winning all around these days.

It does? I would rather argue that portability between desktop OS matters today less than twenty years ago as the niches are carved out, WSL 2 becomes actually usable and the desktop metaphor as a whole declines.

Binary size was a lot more of a problem before customers had to buy 8-16 GB of RAM to run a web browser.

sudo xattr -rd com.apple.quarantine /path/to/binary/or/app

should also allow anything to run without opening settings

Now imagine nuitka and gcc bundled as a standalone cross plateform binary so you can basically compile a cross plaform python exe from any script.

Am I doing something wrong? I tried this on MacOS 12.6.9 and every binary I've tried so far gives a similar "zsh: exec format error":

    $ unzip cosmos-3.0.1.zip
    $ cd bin
    $ ./python -c 'print(2+3)'
    zsh: exec format error: ./python

You need to upgrade to zsh 5.9, or backport the patch I sent them two years ago: https://github.com/zsh-users/zsh/commit/326d9c203b3980c0f841... If you can't upgrade zsh, then the workaround is to say `sh -c ./ape-program` or `bash -c ./ape-program` instead of `./ape-program`.

Awesome, the workaround worked for me. Thanks for the quick reply (and your amazing work on cosmopolitan and the rest)!

Kind of a shame qemu isn't in there. That would be all sorts of fun.

I wonder if llm would work?

Base https://llm.datasette.io/ should work just fine, but it's basically a thin Python wrapper around the OpenAI Python library which then makes HTTP requests to their API.

The bigger challenge would be LLM with plugins. Those add all sorts of extra dependencies to help LLM run models directly - PyTorch, Transformers, llama-cpp-python etc.

I'd be surprised to see "llm install llm-gpt4all" work for example, since that pulls in compiled Python wheels and I'm not sure it would know which architecture to use.

pip install DOES work:

    $ ./python -m pip install httpx
    Defaulting to user installation because normal site-packages is not writeable
    Collecting httpx
      Downloading httpx-0.25.0-py3-none-any.whl (75 kB)
        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.7/75.7 kB 2.7 MB/s eta 0:00:00
    ...
    Successfully installed anyio-4.0.0 h11-0.14.0 httpcore-0.18.0 httpx-0.25.0 sniffio-1.3.0

    $ ./python                     
    Python 3.11.4 (heads/pypack1:65ac8ac, Oct 16 2023, 02:35:05) [GCC 11.2.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import httpx
    >>> httpx
    <module 'httpx' from '/Users/simon/.local/lib/python3.11/site-packages/httpx/__init__.py'>

    $ ls ~/.local/lib/python3.11/site-packages
    anyio
    anyio-4.0.0.dist-info
    h11
    h11-0.14.0.dist-info
    httpcore
    httpcore-0.18.0.dist-info
    httpx
    httpx-0.25.0.dist-info
    sniffio
    sniffio-1.3.0.dist-info

That ~/.local/lib/python3.11/ directory didn't exist until I ran the ./python -m pip install command.

Unfortunately "./python -m pip install llm" didn't quite work, I got this:

    ./python -m pip install llm                   
    Defaulting to user installation because normal site-packages is not writeable
    Collecting llm
      Downloading llm-0.11.1-py3-none-any.whl (36 kB)
    Requirement already satisfied: click in /zip/Lib/site-packages (from llm) (8.1.6)
    Collecting openai (from llm)
      Downloading openai-0.28.1-py3-none-any.whl (76 kB)
        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.0/77.0 kB 31.0 MB/s eta 0:00:00
    Collecting click-default-group>=1.2.3 (from llm)
      Downloading click_default_group-1.2.4-py2.py3-none-any.whl (4.1 kB)
    Collecting sqlite-utils>=3.35.0 (from llm)
      Downloading sqlite_utils-3.35.1-py3-none-any.whl (67 kB)
        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.2/67.2 kB 4.9 MB/s eta 0:00:00
    Collecting sqlite-migrate>=0.1a2 (from llm)
      Downloading sqlite_migrate-0.1b0-py3-none-any.whl (10.0 kB)
    Requirement already satisfied: pydantic>=1.10.2 in /zip/Lib/site-packages (from llm) (1.10.12)
    Requirement already satisfied: PyYAML in /zip/Lib/site-packages (from llm) (6.0)
    Collecting pluggy (from llm)
      Downloading pluggy-1.3.0-py3-none-any.whl (18 kB)
    Collecting python-ulid (from llm)
      Downloading python_ulid-2.2.0-py3-none-any.whl (9.4 kB)
    Requirement already satisfied: setuptools in /zip/Lib/site-packages (from llm) (68.0.0)
    Requirement already satisfied: pip in /zip/Lib/site-packages (from llm) (23.1.2)
    Requirement already satisfied: typing-extensions>=4.2.0 in /zip/Lib/site-packages (from pydantic>=1.10.2->llm) (4.7.1)
    Collecting sqlite-fts4 (from sqlite-utils>=3.35.0->llm)
      Downloading sqlite_fts4-1.0.3-py3-none-any.whl (10.0 kB)
    Collecting tabulate (from sqlite-utils>=3.35.0->llm)
      Downloading tabulate-0.9.0-py3-none-any.whl (35 kB)
    Requirement already satisfied: python-dateutil in /zip/Lib/site-packages (from sqlite-utils>=3.35.0->llm) (2.8.2)
    Requirement already satisfied: requests>=2.20 in /zip/Lib/site-packages (from openai->llm) (2.31.0)
    Collecting tqdm (from openai->llm)
      Downloading tqdm-4.66.1-py3-none-any.whl (78 kB)
        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.3/78.3 kB 41.9 MB/s eta 0:00:00
    Collecting aiohttp (from openai->llm)
      Downloading aiohttp-3.8.6-cp311-cp311-macosx_11_0_arm64.whl (343 kB)
        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 343.5/343.5 kB 3.5 MB/s eta 0:00:00
    Requirement already satisfied: charset-normalizer<4,>=2 in /zip/Lib/site-packages (from requests>=2.20->openai->llm) (3.2.0)
    Requirement already satisfied: idna<4,>=2.5 in /zip/Lib/site-packages (from requests>=2.20->openai->llm) (3.4)
    Requirement already satisfied: urllib3<3,>=1.21.1 in /zip/Lib/site-packages (from requests>=2.20->openai->llm) (2.0.4)
    Requirement already satisfied: certifi>=2017.4.17 in /zip/Lib/site-packages (from requests>=2.20->openai->llm) (2023.7.22)
    Collecting attrs>=17.3.0 (from aiohttp->openai->llm)
      Downloading attrs-23.1.0-py3-none-any.whl (61 kB)
        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 kB 41.9 MB/s eta 0:00:00
    Collecting multidict<7.0,>=4.5 (from aiohttp->openai->llm)
      Downloading multidict-6.0.4-cp311-cp311-macosx_11_0_arm64.whl (29 kB)
    Collecting async-timeout<5.0,>=4.0.0a3 (from aiohttp->openai->llm)
      Downloading async_timeout-4.0.3-py3-none-any.whl (5.7 kB)
    Collecting yarl<2.0,>=1.0 (from aiohttp->openai->llm)
      Downloading yarl-1.9.2-cp311-cp311-macosx_11_0_arm64.whl (61 kB)
        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.3/61.3 kB 427.1 kB/s eta 0:00:00
    Collecting frozenlist>=1.1.1 (from aiohttp->openai->llm)
      Downloading frozenlist-1.4.0-cp311-cp311-macosx_11_0_arm64.whl (46 kB)
        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 46.7/46.7 kB 14.8 MB/s eta 0:00:00
    Collecting aiosignal>=1.1.2 (from aiohttp->openai->llm)
      Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
    Requirement already satisfied: six>=1.5 in /zip/Lib/site-packages (from python-dateutil->sqlite-utils>=3.35.0->llm) (1.16.0)
    Installing collected packages: sqlite-fts4, tqdm, tabulate, python-ulid, pluggy, multidict, frozenlist, click-default-group, attrs, async-timeout, yarl, sqlite-utils, aiosignal, sqlite-migrate, aiohttp, openai, llm
    ERROR: Could not install packages due to an OSError: [Errno 2] ENOENT/2/No such file or directory: '/Users/simon/.local/bin/sqlite-utils'

Yes, so pip works because the Python APE has OpenSSL built-in.

pip install requires modifying the APE, so I end up installing pure-Python libraries as follows:

    mkdir Lib
    ./python -m pip download httpx
    unzip ./*.whl -d ./Lib
    mv python python.com # in case the next step doesn't work
    zip -r ./python.com Lib
    mv python.com python

Installing CPython extensions like this is an unsolved problem, but I think there might be some interesting workarounds possible.

Yes they do!

https://github.com/jart/cosmopolitan/tree/master/third_party...

and

https://github.com/trholding/llama2.c