Hacker News Reader

Comment by Uehreka | Hacker News Reader

So on the one hand, all this Cosmo/ape/redbean stuff sounds incredible, and the comments on these articles are usually pretty positive and don’t generally debunk the concepts. But on the other hand, I never hear mention of anyone else using these things (I get that not everyone shares what they’re doing in a big way, but after so many years I’d expect to have seen a couple project writeups talk about them). Every mention of Cosmo/ape/redbean I’ve ever seen is from Justine’s site.

So I’ve gotta ask: Is there a catch? Are these tools doing something evil to achieve what they’re achieving? Is the whole thing a tom7-esque joke/troll that I don’t get because I’m not as deep into compilers/runtimes? Or are these really just ingenious tools that haven’t caught on yet?

View on HN

APE works through cunning trickery that might get patched out any day now (and in OpenBSD, it has been).

Most people producing cross-platform software don't want a single executable that runs on every platform, they want a single codebase that works correctly on each platform they support.

With that in mind that respect, languages like go letting you cross compile for all your targets (provided you avoid CGO) is delightful... but the 3-ways-executable magic trick of APE, while really clever, doesn't inspire confidence it'll work forever, and for the most part it doesn't gain you anything. Each platform has their own packaging/signing requirements. You might as well compile a different target for each platform.

Wasn't elf format modified by upstream to accomodate for cosmo? That makes it kinda official. Still hard to see a use case for it. If you want everyone to be able to run your program, just write a web app, a win32 program, or a java applet. 20 years old java applets still run on modern JVMs.

You can't reasonably assume the end user system has a system JVM installed (they probably don't, in fact) so they're not really an alternative to a fat binary -- if you can install dependencies, you can just pick a single-target binary while you're at it.

but you can provide that JVM for different plattforms in a zip file/installer.

The more difficult problem with cross plattform apps are always native look and feel and calling native apis in an idiomatic and performant ways.

Justine has similar claims about POSIX allowing binary in shell scripts now

> This is an idea whose time has come; POSIX even changed their rules about binary in shell scripts specifically to let us do it.

https://justine.lol/cosmo3/

> Jilles Tjoelker from the FreeBSD project played an instrumental role in helping me to get the POSIX rules changed to allow binary in shell scripts, which is what made this project possible.

https://justine.lol/ape.html

but that's not true, as recently discussed here: https://news.ycombinator.com/item?id=41636569

A web app is, well, a web app. Many things don't fit this format, e.g. command line tools.

A Win32 program will not run out of the box on either Linux or macOS. Neither will a Java app.

The nice thing about Cosmopolitan is that it "just works" as far as end user is concerned. But without firm support from the OSes involved, it is inevitably a hack with questionable long-term stability prospects.

What we really need is some kind of standardized low-level (think LLVM bitcode or wasm) architecture- and platform-agnostic binary format that can be JIT-compiled to the same native code that a C compiler would have produced from the source. And that is supported by all major OSes out of the box.

Yes that's exactly what we need. And we'll be laughing to the bank when we bundle a browser toolbar with adware into its installer ten years down the road. Oh wait Java already did this. OTOH Cosmopolitan gives you complete autonomy. You don't need a JVM to run a simple native command line program on multiple OSes and I proved that.

JVM bytecode is obviously not what I was talking about - it is neither low-level (if you can't efficiently compile the entirety of C into it, it's not good enough for this purpose), nor did it ever have universal first-class OS support.

And stuff like https://github.com/jart/cosmopolitan/issues/1263 shows that just because it works today, it doesn't mean that it'll work tomorrow. What's the guarantee that the same won't happen on Linux or macOS eventually?

Don't get me wrong, APE is a hell of a hack, and I love it for that. But it can't be a truly stable platform without guarantees from the underlying OSes to keep all of that machinery working the same way. And if we could actually get them to agree on something like that, why not get them to agree on an actual portable executable format that is future-proof also wrt new CPU architectures?

(Note, by the way, that this is orthogonal to the whole notion of fat binaries that contain arch-specific code. That is still a useful optimization to have in practice even if one has portable bytecode, alongside said portable bytecode.)

I agree. As a user you should have those guarantees. Here's how I'm helping you get them.

- With Windows, APE is just PE and WIN32 so those are already guaranteed by Microsoft.

- With Linux, I've been working with kernel maintainers to build consensus around around a patch https://justine.lol/ape.patch that will let your APE binaries be loaded by binfmt_elf. This will provide you with a guarantee that they'll still work, even if someone has for instance registered a binfmt_misc entry like WINE that's intercepting MZ executables.

- With FreeBSD, I'm also working to build consensus here around another patch, that will help the kernel load APE natively as a normal ELF executable. That means you won't need to rely on the system shell to load your program into memory.

- I've been writing a specification for APE that documents this file format, its ABI, and its historical encodings. https://github.com/jart/cosmopolitan/blob/master/ape/specifi...

One day in the future, when we have the support of major OSes for the native loading of portable binaries, we'll look back at the APE shell script bootstrapping process similar to how we look back at self-extracting zip archives. When Phil Katz first invented PKZIP, it was useful to prepend the software to each archive. But nowadays it isn't necessary, since all OSes come with support for PKZIP built in.

I rather suspect that Apple will not play along.

That said, a fat binary format that works for both Windows and Linux is already very useful.

If we earn the support of Linux and FreeBSD then we might stand of chance. Apple might ask for a better way to secure ape executables, which could be potentially exciting, since it'd be nice to have a vendor neutral solution to code signing.

> But without firm support from the OSes involved, it is inevitably a hack with questionable long-term stability prospects.

Case in point:

https://github.com/jart/cosmopolitan/blob/master/libc/sysv/s...

The system call numbers of all the unixlikes are bitwise packed into a single number. There is exactly one of those columns which is stable: the Linux one. Everything else is not part of the binary interface of their respective operating systems.

I've written about how Linux is special in this regard:

https://www.matheusmoreira.com/articles/linux-system-calls

It's a neat hack but I'm afraid it's in grave danger of falling victim to the Darth Vader of OS ABI stability.

https://lwn.net/Articles/806870/

  Program to the API rather than the ABI.
  When we see benefits, we change the ABI
  more often than the API.

  I have altered the ABI.
  Pray I do not alter it further.
  
  -- Theo de Raadt

> Most people producing cross-platform software don't want a single executable that runs on every platform

They don't? Having one file to download instead of a maze of "okay so what do you have" is way easier than the current mess. It would be very nice not to have to ask users what platform they're on, they just click a link and get the thing.

Trade offs. While there is a point to that, memory, bandwidth and storage are not free. Users with constrains will want a smaller executable. Of course all 3 are pretty cheap these days so more and more you can get by with just one does it all executable and nobody will care about the downsides, but lets not lose track of them.

Having the browser get the right URL for the download button is by far the most trivial issue when it comes to multi-platform support.

> Most people

We'll I'm used to not being most people, but I'd much rather be able to produce a single identical binary for my users that works everywhere than the platform specific nonsense I have to go through right now. Having to maintain different special build processes for different platforms is a stupid waste of time.

Frankly this is how it always should have worked except for the monopolistic behavior of various platforms in the past.

The binary is only one part of the puzzle (and largely solved by WSL). Installation/uninstallation and desktop integration is just as much of a hassle.

I don't get the argument, if the binary part is solved by WSL it isn't useless is it? Otherwise why would MS invest so much resources into it?

WSL can do a lot more than just run Linux binaries.

I don't think you can reasonably assume that people have WSL set up on Windows for the purposes of shipping desktop software. Nor does it cover Mac.

Well, there's orb on mac which is in some ways better/faster than WSL

> that works everywhere

This whole discussion is about pointing out that it kinda doesn't.

> With that in mind that respect, languages like go letting you cross compile for all your targets (provided you avoid CGO)

Even that is not a big deal in most of cases, if you use zig to wrap CC: https://dev.to/kristoff/zig-makes-go-cross-compilation-just-...

Does this still work? The article is from 2021 but when I last tried it this year, Go appeared to (newly) depend on headers that Zig doesn't need and thus it doesn't work. The Github issue was something like "yeah, we don't need those, so I guess Go doesn't work anymore". Without the actual error message I can't find the issue, however, so maybe I imagined this.

Yes, I use it in my PKCS#11 client glue code.

I believe these are the issues: https://github.com/golang/go/issues/52690 https://github.com/ziglang/zig/issues/14989

what worked for me for cross-compiling go:

https://github.com/elastic/golang-crossbuild docker images for macos and windows (not linux though)

https://github.com/rust-cross/rust-musl-cross docker images for linux (yes, it's possible to use the same one for go, you just need to put go binary there) so it's static with musl libc and sure to not have some weird issues with old/new libc on old/new/whatever linuxes

(it might be over the top and the zig thing might effectively do the same thing, but I never got it to compile)

oh and apple-codesign rust cargo to get through the signing nonsense

> Is there a catch?

I am only speaking for myself here. While cosmo and ape do seem very clever, I do not need this type of clever stuff in my work if the ordinary stuff already works fine.

Like for example if I can already cross-compile my project to other OSes and platforms or if I've got the infra to build my project for other OSes and platforms, I've no reason to look for a solution that lets me build one binary that works everywhere.

There's also the thing that ape uses clever hacks to be able to run on multiple OSes. What if those hacks break someday due to how executable formats evolve? What if nobody has the time to patch APE to make it compatible with those changes?

But my boring tools like gcc, clang, go, rust, etc. will continue to get updated and they will continue to work with evolving OSes. So I just tend to stick with the boring. That's why I don't bother with the clever because the boring just works for me.

If the executable format changes then it would affect more than just ape/cosmopolitan. All of the old binaries would just stop working.

I am not worried about the scenario where the executable format changes breaks old binaries. That is never going to happen. No OS worth its name is going to do that.

I am worried about the scenario where the executable format changes just enough to break APE binaries. Like I said, APE uses very clever hacks to make itself work on multiple OSes. It is conceivable that executable formats change just enough that it breaks the clever hacks in APE binaries without breaking old ordinary binaries.

Mozilla llamafile uses it. Bundles model weights and an executable into a single file, that can be run from any cosmo/ape platform, and spawns a redbean http server for you to interact with the LLM. Can also run it without the integrated weights, and read weights from the filesystem. It's the easiest "get up and go" for local LLMs you could possibly create.

Mozilla’s llamafile is primarily developed by jart. I wouldn’t view it as anyone else actually using cosmo/ape

Yes I should have mentioned that, even when I originally interpreted the OP to be asking about other projects using it, not necessarily other people. But they almost certainly meant other people not other projects, so my point is kind of void.

Although in fairness, the Mozilla org choosing Justine/cosmo is basically the kind of sign-off from a 3rd party that OP was looking for.

reposting my comment from another time this discussion came up:

"Cosmopolitan has basically always felt like the interesting sort of technical loophole that makes for a fun blog post which is almost guaranteed to make it to the front page of HN (or similar) purely based in ingenuity & dedication to the bit.

as a piece of foundational technology, in the way that `libc` necessarily is, it seems primarily useful for fun toys and small personal projects.

with that context, it always feels a little strange to see it presented as a serious alternative to something like `glibc`, `musl`, or `msvcrt`; it’s a very cute hack, but if i were to find it in something i seriously depend on i think i’d be a little taken aback."

But what's actually bad about it?

The problem with this take is that it is not grounded in facts that should determine if the thing is good or bad.

Logically having one thing instead of multiple makes sense as it simplifies the distribution and bytes stored/transmitted. I think the issue here is that it is not time tested. I believe multiple people in various places around the globe think about it and will try to test it. With time this will either bubble up or be stale.

Mozilla has a project called Llamafile (https://github.com/Mozilla-Ocho/llamafile) that's based on Cosmopolitan libc. And they do regularly publish popular models repackaged in that format on Hugging Face: https://huggingface.co/models?search=llamafile.

Whether that in turn has any practical use beyond quickly trying out small models is another question.

> So I’ve gotta ask: Is there a catch? Are these tools doing something evil to achieve what they’re achieving?

it's not that complicated; they're fat binaries (plus i guess a lot of papering over the differences between the platforms) that exploit a quirk of tshell:

> One day, while studying old code, I found out that it's possible to encode Windows Portable Executable files as a UNIX Sixth Edition shell script, due to the fact that the Thompson Shell didn't use a shebang line.

(https://justine.lol/ape.html)

so the answer is simple: i can't think of anyone that wants to ship fat binaries.

Anyone focused on customer experience wants to ship a binary that just works, without customers having to know what a fat binary even is.

I don't think macos will allow you to run a downloaded cosmo binary without going into security settings and enabling it. That's not an experience I'd want my customers to have personally which means if you care about mac normies, this isn't a viable approach.

I don't know what magic is involved, but I was able to download eg https://cosmo.zip/pub/cosmos/bin/md5sum and run it in a terminal without having to do run 'xattr -d com.apple.quarantine' on it.

Unless the customer is one who has issues with large downloads. Not everyone has fiber to the home with massive storage on modern computers. Some people have the entry level systems, settle for slow internet. Often they are in poor or "third world" places which means maybe you don't care about them, but they exist.

Fat binaries are fine. Electron is a fat binary in a different sense.

Given how fat modern websites are, compassion for the bandwidth deprived seems rare.

Very much. Which is really a problem for those who do have bandwidth limitations. Or even low end phones / old computers. Or even new computers that are not very powerful.

Correct me if I'm wrong, but I always thought Apple universal binaries are fat binaries? So why did Apple build this ability if nobody wants it?

> So why did Apple build this ability if nobody wants it?

did you miss the part where they transitioned from x86 to arm64 in 2020?

I don't get your point. People were arguing that one doesn't need fat binaries because cross compiling and having different binaries is fine. Apple clearly thought differently when they transitioned from x86 to arm (not their first architecture transition either).

So now you're saying because apple finished the transition fat binaries are useless again? What about other platforms?

> finished the transition fat binaries are useless again?

yup that's exactly what i'm saying.

> apple finished the transition fat binaries are useless again?

Is the transition really finished? I'm writing this on a x86_64 Macbook with a browser that is distributed as x86_64/arm64 universal binary.

And PowerPC to x86 before that.

Apple's fat binaries were never very "universal". They put in a codepath where macOS detects and handles macOS-arm-and-x86_64 binaries. That is something where they can support both variants, and themselves in control. Cosmopolitan libc is a pile of hacks, compared to that, more comparable to somebody making Linux boot on a 4004 CPU than what Apple did.

> Is the whole thing a tom7-esque joke/troll that I don’t get because I’m not as deep into compilers/runtimes? Or are these really just ingenious tools that haven’t caught on yet?

If I went up to you in 2008, and said, hey, lets build a database that doesn't do schemas, isn't relational, doesn't do SQL, isn't ACID compliant, doesn't to joins, has transactions as an afterthought, and only does indexing sometimes, you'd think I was trolling you. And then in 2009, Mongodb came out and caught on in various places. So only time will tell if these are ingenious tools that haven't caught on yet. There's definitely a good amount of genius behind it, though only time will tell if it's remembered in the vein of tom7's harder drives or if it sees wider production use. I'll say that if golang supported it as a platform, I'd switch my build pipelines at work over to it, since it makes their output less complicated to manage as there's only a single binary to deal with instead of 3-4.

Tbh I don’t know that there is a catch except that there is one core person behind whole project. I like that there are fresh ideas in the C space.

Last year I needed to make a small webapp to be hosted on a Windows server, and I thought RedBean would be ideal for it. Unfortunately it was too buggy (at least on Windows).

I don't know whether RedBean is production-ready now, but a year and a half ago, that was the catch.

Give the latest nightly build a try: https://cosmo.zip/pub/cosmos/bin/ Windows has been a long hard march, but we've recently hit near feature completion. As of last month, the final major missing piece of the puzzle was implemented, which is the ability to send UNIX signals between processes. Cosmopolitan does such a good job on Windows now that it's not only been sturdy for redbean, but much more mature and complicated software as well, like Emacs, GNU Make, clang, Qt, and more.

Most people aren't writing C as far as I know. We use Java, C#, Go, Python etc, some lunatics even use Node.

We generally don't care if some mutex is 3x faster than some other mutex. Most of the time if I'm even using a mutex which is rare in itself, the performance of the mutex is generally the least of my concerns.

I'm sure it matters to someone, but most people couldn't give two shits if they know what they're doing. We're not writing code where it's going to make a noticeable difference. There are thousands of things in our code we could optimize that would make a greater impact than a faster mutex, but we're not looking at those either because it's fast enough the way it is.

What a pointless contribution

I came here to write this. In such a good discussion thread, this was such a sore thumb

Reply on news.ycombinator.com