Hacker News Reader

Comment by dataangel | Hacker News Reader

Is there any practical purpose for APE? Like who is shipping single executables that need to run on N OSes?

I use an APE executable as an agent for communicating with remote hosts in the Logfile Navigator (https://lnav.org). While lnav itself is not built as an APE, the agent built into itself is. That agent is transferred to the remote when the user wants to read logs on that host. This way, there is no extra step to determine the type of OS or building in multiple versions of the executable. Here's a short blog post on the subject:

https://lnav.org/2021/05/03/tailing-remote-files.html

My biggest problem with APE: not portable enough, in the sense that when people adopt APE, they lock themselves into less portability than they might have supported otherwise. I live in an ARM world and because of APE you only offer Intel x64 binaries. All new Macs are ARM64, Raspberry Pi models are a mix of ARM32 and ARM64 (mine is ARM64), and I use exclusively Graviton instances in AWS. Running x64 binaries with QEMU/Rosetta emulation on an ARM system is a cute trick but not something I want to do for a piece of infrastructure. Raspberry Pis are barely fast enough to run native binaries. Consider offering APE for x64 but then still producing ARM binaries the old fashioned way. APE is pushing the idea that x64 is a good bytecode in a world that is moving from x64 to ARM. I'd like to see the project accept that ARM exists and produce an ARM version of APE, maybe with some jart flourish--can we make a cross-platform and cross-architecture fat binary?

> APE is pushing the idea that x64 is a good bytecode

And x86-64 is a particularly bad bytecode, for one main reason: its strong memory ordering rules, compared to nearly every other architecture. IIRC, simple things like doing a store in x86-64 have an implicit release barrier, while other architectures can freely reorder the stores unless there's an explicit barrier. This means that a x86-64 emulator on for instance ARM has to either be single-threaded (and would still have issues with shared mmap regions), have special hardware support for x86-compatible memory ordering (as found for instance on recent Apple ARM CPUs, and used for their x86-64 emulation, but not common elsewhere), or add explicit barriers everywhere (which kills the performance).

The only real advantage of x86-64 as a bytecode would be that it has less registers than most other architectures (only 16 registers, while other 64-bit architectures usually have around 32 registers), which allows a 1:1 mapping of the registers on the emulator while still leaving plenty of them free for temporary use.

> Consider offering APE for x64 but then still producing ARM binaries the old fashioned way.

The recent version of cosmopolitan generates ARM binaries for Linux and MacOS (https://github.com/jart/cosmopolitan#arm; mode aarch64). There is also blink that provides the x86-64 emulation layer for (APE and other) binaries on a variety of platforms (https://github.com/jart/blink).

I didn't know about that, thanks! It gives me hope that the final goal (fat binary, native ARM and native x64, all platforms on both architectures) is within reach. It looks like there are still some hiccups (the APE loader on M1) but they are much further along than I thought they were. Excellent!

Speaking about fat binaries; @jart posted this a day ago in the #ape channel on project discord:

Just created my first ever fat ape executable. I ran `apelink.com -o fat-ape-binary.com o/tiny/examples/hello2.com.dbg o/aarch64-tiny/examples/hello2.com.dbg` to create an executable that runs on Linux+OpenBSD+NetBSD+FreeBSD on x86_64+aarch64.

Java/Closure/Kotlin programmers. .Net programmers. JS, Python, Ruby, Erlang, every VM-based language programmers. Statically-compiled programs are under the hood of everything, but at the application level in the minority.

APE is a big “Who needs a VM if an AOT compiled language as low level as C can compile once and run universally?”

Portability is not the reason you see VMs everywhere.

I think it is a big part of it.

If libc is the only leaf dependency.

Which is true for a lot of useful tools that run in a terminal (for many coders this is pretty much everything except an IDE and a browser).

If their world is ISO C and nothing else, quite a boring one actually.

Not even a bit of colour, as terminal escape sequences assume a POSIX host environment, and will throw up gibberish if not.

Nah, terminal colors via escape sequences work fine on Windows, even on the "traditional" cmd.exe. I use that all the time in cross-platform Python and Typescript cmdline tools (cmd.exe is probably limited to 16 colors though, but that's just how it is).

Windows isn't the only non POSIX one out there...

And it was only one example, another one would be any kind of networking, as it isn't part of ISO C.

There seems to be a standard socket API in the Cosmopolitan clib though:

https://github.com/jart/cosmopolitan/tree/master/libc/sock

..and that also seems to have WinSock support:

https://github.com/jart/cosmopolitan/blob/master/libc/sock/c...

(also Windows is probably the only non-POSIX OS that matters - and APE is obviously only portable to systems it specifically supports, it can't support systems it doesn't even know about, no matter if it is a "POSIX system" or not).

Yeah, but that is going beyond ISO C, and diving int POSIX C LIB, which is the point I was making already.

I can imagine it making it slightly easier to distribute executables from e.g. maven repositories (where having multiple artifacts for the same id/version combination is cumbersome). I think PyPI might have a similar issue?

Also cross-compiling is painful; a lot of small OSS projects that don't have a build farm with all of the different BSDs etc. to hand simply don't ship binaries for those. Even e.g. kubectl has a binary for Linux but not for any of the BSDs (unless you count OSX). So this seems like an easy win for those.

I'm trying to release future side projects that way. I even build a Redbean executable for any client-only SPAs. The idea of this stuff just working on so many systems is amazing.

Problem with this is that self-modifying code will likely trip security blocks on most platforms, which is probably more annoying for users than having to download separate executables.

If the person needs to run with "security blocks" up, it's statistically unlikely that the same person can randomly download the specific and potentially outdated version of a dependency you need for the code to run in the first place.

I have a shader compiler which would benefit from a platform agnostic, ready-to-run executable format.

It's too big to be build ad-hoc on the user's machine (contains various big 3rd party C++ dependencies, and takes between 2 and 20 minutes to build, depending on the hardware).

Needs to run on macOS x86-64+ARM, Linux x86-64 (ARM would be nice too of course), and Windows x86-64.

I'm currently looking into WASM, but that needs an installed WASI runtime.

A long time ago I wanted an Ansible equivalent that worked by injecting a bootstrap VM, then sending configuration code to run on the remote node, rather than chatting back and forth over an SSH tunnel. A lot of the slowness of applying config was that chattiness, and it would have been a lot faster to avoid it.

I have no idea if there is a tool that actually works like this - I remember an exploit tool that did something similar, but it wasn't quite right - but making that initial bootstrap VM and a useful stdlib an APE executable would also have avoided the "there needs to be a working Python at the other end" issue which persisted in tripping me up for far longer than it should have.

i've been asked this hundreds of times in my life, by customers, bosses, etc...

But they don't know or care the difference between the code being in the same binary or not

But they know and care that "it doesn't run" even if they don't know how and why.

I mean, yes they care, they want to be able to put the executable in a usb key or an email and that it runs on any computer they would plug it into

Storing executables on file systems shared across operating systems or architectures gets a lot easier if you don't have to have special paths for each os+arch combination.

Doesn’t the executable get overwritten after first invocation?

I don't think so.

Only if you specifically run it with the `--assimilate` flag.

I've seen this solved before by giving each combo a different mount with the same path.

In vscode extensions that rely on an external binary to do most of the job?

It's much more useful for greenfield software. Mad props to whoever wrote the gcc patch just to get something like this work some of the time, but rebuilding existing software is probably a fool's errand. There's a good reason you see things like the rpm package format include not only kernel but distro. Running on "Linux," BSD, Windows, or MacOS is one thing, but Linux is a bunch of different OSes, not one. A whole lot of software out there relies upon assuming you're using a particular DNS provider, filesystem hierarchy that may or may not be FHS-compliant. When I went down a rabbit hole a few years back trying to extend Linux from Scratch to include a bunch more packages, I found that GHC won't build if the file "/etc/os-release" doesn't exist. The file can be empty, but it has to exist.

Someone mentioned busybox elsewhere, which sure, will build with this if you disable a whole bunch of stuff, but busybox is supposed to be the minimal utilities needed to run something close to POSIX-compliant, which obviously assumes your system is a POSIX system. It also includes most of the stuff usually provided by util-linux, which obviously won't run except on Linux. If your software is doing something like trying to read from particular device files to get hardware info from the kernel, sometimes that will work on both Linux and BSD (and even Mac since it's kind of a BSD), but it definitely won't work on Windows.

For software that uses the network, how do you query DNS? If you're calling getaddrinfo, I guess Cosmo libc will do some magic for you to make sure that works everywhere, but a lot of software not written in C is doing something not quite that easily made universal, like just assuming /etc/resolv.conf exists and reading it, which definitely won't work on Windows, or using the native platform APIs, which will only work on one platform.

If you distribute desktop software, I would say good luck. Are you going to comply with the XDG? Well, now I guess it will "work" on Windows, but you're not complying with Windows conventions where you're supposed to put stuff in App\RoamingLow or whatever the hell that I would never remember without looking it up every time. Don't try storing data in the registry, because now it will only work on Windows.

Does your software? It better be present on all systems then. Do you query any environment variables? I don't think LC_TIME and LC_COLLATE are provided anywhere but Linux usually. The XDG_ variables are only Linux. Does Windows have PAGER? I don't even know. I'm pretty sure PATH, USER, and HOME work everywhere.

A truly actually portable executable (TAPE) would require far more than an executable file format that can successfully load into memory and execute its first instruction on any platform. There's a reason vendors just write software that uses a browser's JavaScript engine as a platform instead of the OS's native runtime.

EDIT: I don't know if this is ironic, but I guess consider your choice of programming language, too. Go became popular in part because of the static linking, single executable thing, but it inherently can't be cross-platform since it doesn't use the C library and tries to make all calls into the kernel on its own by keeping track of the syscall table provided by every OS it supports. Well, they're still different on every OS, so an executable that works on Linux will not work anywhere else and vice versa.

> I'm pretty sure PATH, USER, and HOME work everywhere.

Out of those 3 only PATH works on Windows.

Windows has a USERNAME environment variable, but depending on your needs you may need to combine it with the USERDOMAIN variable to get the complete username, %USERDOMAIN%\%USERNAME%.

Windows has a HOMEPATH environment variable but it contains only the directory for the home folder. Another variable, HOMEDRIVE, contains the drive the home directory exists on, so the complete home location is %HOMEDRIVE%\%HOMEPATH%. In powershell a read-only HOME variable is set pointing to %HOMEDRIVE%\%HOMEPATH%. The home directory is supposed to store user files. User specific applications, OS components, customization and configuration data should be stored in the USERPROFILE directory. This is very frequently but not always the same as %HOMEDRIVE%\%HOMEPATH%.

In what circumstance would USERPROFILE differ from HOMEDRIVE\HOMEPATH?

When users home folders are redirected to a network share. In that circumstance it is also possible that the user's home directory might be HOMESHARE\HOMEPATH instead of HOMEDRIVE\HOMEPATH. This will be the case if the network share isn't mapped to a drive letter. Really HOMESHARE should be checked for and only if it doesn't exist HOMEDRIVE used. I should have pointed that out in my original comment but it was made pre-coffee.

Folder redirection is used often in conjunction with remote desktop services, virtual desktops or just to centralize file storage for ease of backup.

Reply on news.ycombinator.com