Show HN: Redbean – Single-file distributable web server | Hacker News Reader

Show HN: Redbean – Single-file distributable web server

justine.lol

1,998 pointsjart249 comments

I just spent 30 minutes reading about this. I'm so shocked that I'm logging in to comment after 5 years of lurking.

Justine has built a c library that allows you compile a binary once and have it run it on any os or baremetal. The SAME binary. Quite frankly, that sentence doesn't even make sense to me.

Check out https://storage.googleapis.com/justine/cosmopolitan/index.ht...

As far as I'm concerned, this is literal magic. Look at the magic numbers: https://github.com/jart/cosmopolitan/blob/37a4c70c3634862d8d...

I could go on, but there's no binary portability comparison with any other language. And she has made some pretty neat optimizations.

Back in the day, I saw some pretty neat stuff with the ELF format, but this takes the cake.

Wow.

Edit: I'm editing because this is just so bloody absurd.

https://storage.googleapis.com/justine/printimage.html

$ ./printimage.com someimage.jpg

Like wow. And also video.

https://storage.googleapis.com/justine/printvideo.html

I'm struggling to put my shock into words. I've been around.

There's engineering. There's academia.

But this falls into straight-up wizardry.

I agree, I got to the page on actually portable executable, and had to read it a few times to make sure I was grokking it properly. This is just sheer cleverness, THIS is the stuff that should be on hacker news!

For real.

It's hard to grok cause there are no words to describe it.

When people say that something is cross-platform, they usually mean a) that the software will build on multiple platforms b) there is some sort of vm which runs the executable (jvm, beam, wasm)

This is the SAME binary. Running everywhere. Could be super-useful as an archival format for mission critical code. sqlite comes to mind.

Maybe it's late at night, but I'm struggling to find a suitable word for the project that is better than portable or cross-platform.

Literally, I can't find a better word than the project name - "actually portable executable."

@author - if you're reading this please set up something for a few bucks a month on your github sponsors. I don't think I have any use for the library, but this is so outrageous it deserves something more than imaginary internet points.

Author here. Thanks! I've added a new sponsor tier per your request. https://github.com/sponsors/jart

Do you have a direct paypall? Im not a fan of subscription services but Id like to support this.

I do indeed have PayPal: jtunney@gmail.com Thank you for your support!

> THIS is the stuff that should be on hacker news!

If interested, see previous discussions of APE:

https://news.ycombinator.com/item?id=24256883

https://news.ycombinator.com/item?id=25556286

Thanks for the links. The discussion on the first one is interesting, it wasn't clear to me from the original doc that the UNIX version needs to overwrite itself to get to the ELF header. That's a bit unfortunate, because it means you can't share the program again after you ran it the first time on BSD or Linux. Still very clever, though. I imagine there could be a "repackage for distribution" switch added to put the MZ header back again.

I did the research and what we've accomplished here, while imperfect, is the best of all worlds solution. I like to think of it as an installer that takes a few microseconds, because it only needs to change 64 bytes. I've been considering adding a CLI flag where you can say ./foo.com --restore and it'll put the original 64 byte header back in place. Perhaps one day we can change the Linux, BSD, and XNU kernels so they can recognize the APE executable format. But until that happens we've got a great shell script hack that's now required by POSIX which is exceedingly fast and works on pretty much all systems stretching back decades.

Why didn't you copy to a tmpfs and overwrite that version instead?

Seems more likely to have installed programs on read-only media and a little tmpfs, instead of a writable binary and no tmpfs.

Do you have any information on some of the other strategies you researched?

I really don't understand internals ibut I've often downloaded cygwin exe file for commands like grep, tail, etc.. Are you telling me that is no longer required?

On linux, I downloaded the printvideo binary and played a friggin crab video on my terminal after just a chmod. I don't have any other OS to confirm, but it sure looks like it if someone compiles/links coreutils against this.

Everything she does has this level of jaw-dropping amazingness. Between her and Fabrice Bellard i don't know many people who consistently get my chin to hit the table.

I hope that Cosmopolitan becomes mainstream, so much more software could have that It Just Works quality.

"Jaw-dropping" is definitely a term I'd use to describe some of her more infamous tweets and blog posts, too.

Please don't. Resisting extrinsic provocation is key to thread quality.

Understood (with my apologies, I mistakenly assumed it was on topic).

FWIW, I like and agree with the things I was referencing, I didn't mean it in any way as a jab or provocation. I shall endeavor to do a better job staying on the topic of the article in question.

It was on topic, your mistake was to write a comment that comes off snarky and political of you fail to read it with positive intent.

Mike Pall for LuaJIT

Agreed, this is the best programming-related thing I've seen on the internet in a long time. It reminds me of when I was a teenager, excited to become a computer programmer. Then I grew up and joined the real world workforce and it was all far less exciting than I imagined.

The coolest thing about this Actually Portable Executable is that once there's a compiler and linker built with it, I can play around with writing C on Windows without having to faff about with WSL or MinGW or learning what makes MSVC different from the C I learned in university and coded in my first job before I became a Java/JavaScript/Python/etc programmer. When I discovered Go I thought I had discovered the better C that just worked everywhere, but the idea of having plain old C that also just works everywhere is very appealing.

I like Go for the same reason, but I have an irrational fear that someday I might be in a life or death situation and should program my way out using a bare bones computer without Internet; So I keep tabs on C occasionally.

αcτµαlly pδrταblε εxεcµταblε has now piqued my interest, So I might actually indulge in C programming for fun again after all. Thanks Justine.

I think it's really cool project; but could you explain why it's important that a binary works everywhere for you? I don't really get the enthusiasm for this sort of thing.

Maybe I have a blind spot or something or I've just been in Unix-land for too long.

I think the UNIX thing is the blind spot. People who run Linux or BSD don't seem so bothered to recompile for their own platform, safe in the knowledge that it will almost certainly just work.

But on Windows, trying to get open source software written in C to compile has been a pain for decades. There have been some attempts to make it work (notably Cygwin, and now MSYS2), but it can still be challenging, especially with programs that were never designed to be compiled for non-UNIX (or non-Linux!) targets.

Having a C library that is designed from the ground up to target multiple operating systems, and an executable format that means the first compile is the only compile, that takes away a lot of toil that programmers normally have when trying to build their own abstractions around each platform's C library. I guess from UNIX land you could see the benefit as removing the need for autoconf or platform ifdefs.

Will this work if your code uses the C++ standard library ? Or is this just for pure C ?

The are a few options but all require some effort https://github.com/jart/cosmopolitan/issues/27

That malware for OS X that does nothing (so far) from the news earlier this week also has one multi-arch binary for all modern Apple notebook architectures (AMD64 and ARM64)

In MacOS this is really just a feature of using Mach-O’s multi architecture abilities, they call this universal binaries.

Agreed. This single sentence at the end of αcτµαlly pδrταblε εxεcµταblε page:

> I believe the best chance we have of doing that [writing software that stands the test of time with minimal toil], is by gluing together the binary interfaces that've already achieved a decades-long consensus, and ignoring the APIs.

...it's a kind of thought I don't think I could ever come up with. Mind blown.

You go to war with the army you have.

Strong agree; someone needs to give her like a million dollars a year salary and unlimited resources to explore whatever she wants for life, just to see what she comes up with.

one millliiiooonnn dollars.... Hey Dr Evil (Austin powers reference), I think one billion is okay this year and age :)

but yea I agree

Wow, that cosmopolitan C library is absolutely awesome. The ABI specialization is extremely interesting. Never seen anything like that before. I thought compilers did that. I don't understand why they couldn't properly optimize memcpy.

I wonder how it handles system calls. Let me check the source...

https://github.com/jart/cosmopolitan/blob/master/tool/build/...

> void OpSyscall(struct Machine *, uint32_t);

Pretty interesting... Need some time to figure out how it works.

The system call entry points:

https://github.com/jart/cosmopolitan/blob/master/libc/sysv/s...

It uses a jump slot technique which is interesting:

https://github.com/jart/cosmopolitan/blob/master/libc/sysv/m...

System call numbers:

https://github.com/jart/cosmopolitan/blob/master/libc/sysv/c...

https://github.com/jart/cosmopolitan/blob/master/libc/sysv/s...

Looks like it packs the system call numbers of all operating systems into a single constant.

For some reason there are some dedicated implementations of Linux system calls using inline assembly rather than the entry point:

https://github.com/jart/cosmopolitan/tree/master/libc/linux

More related files:

https://github.com/jart/cosmopolitan/blob/master/libc/calls/...

Also funny how it calls Linux "GNU/Systemd"!

"...it calls Linux "GNU/Systemd""

Yeah, it would have taken less bits to use GNU/POS instead, because it ain't Linux anymore.

Poettering Operating System.

It's funny because the high-level introductions to C I would read as a kid before I really knew much about computing would almost always start out by explaining that one of the biggest strengths of C was its portability, and how it allowed you to write the same code that worked on every platform. Of course I came to learn that's true only in a sense that has no connection to practical reality, this looks like an exception.

Impressive.

Could you please explain to someone from the web domain with very little experience with compiled C programs why this is significantly better than, say, distributing a python program? Python runs on all the platforms mentioned. Or alternatively requiring compilation from source?

The python script/program ultimately depends on a Python interpreter which currently has to be distributed as a different binary for every platform.

So in the end, the difference is purely where the platform-differentiated compilation happens for VM & interpreted languages like Java & Python.

This would instead allow you to ship a single Python executable that simply works everywhere. Whether with a bundled-in script/program, or just the interpreter.

While this is absolutely amazing, and I'm still trying to see if I'm even able to grok the how..

But, the paranoid in me is now curious/afraid if this approach can/will lead to multi-OS malware?

That sort of malware already exists. This does simplify making it by a lot, because now it can much more easily ensure its portability without external communication.

I absolutely love reading Justine's code. It comes up from time to time here and it just makes me happy. It reminds me of the passage from "Programming Sucks" by Still Drinking [1]

  Every programmer occasionally, when nobody’s home, turns off the lights, pours a glass of scotch, puts on some light German electronica, and opens up a file on their computer. It’s a different file for every programmer. Sometimes they wrote it, sometimes they found it and knew they had to save it. They read over the lines, and weep at their beauty, then the tears turn bitter as they remember the rest of the files and the inevitable collapse of all that is good and true in the world.

  This file is Good Code. It has sensible and consistent names for functions and variables. It’s concise. It doesn’t do anything obviously stupid. It has never had to live in the wild, or answer to a sales team. It does exactly one, mundane, specific thing, and it does it well. It was written by a single person, and never touched by another. It reads like poetry written by someone over thirty.

Her work is always just a little bit trippy in a good way haha.

[1] http://stilldrinking.org/programming-sucks

This is too real. Except at this point, after 25 years I have probably ten of these.

I've been thinking of putting together github repo which is just an index out to people's snowflakes. If I do, I'll reach out.

Do it. I’d enjoy perusing it.

Yes, please.

The short description is a bit too short; this is mind blowing!

Redbean is a portable, single file executable webserver which also acts as a zip-file for the static content it hosts, and runs on Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS without any recompilation...

You can manage the static content using standard pkzip tooling! Now that's what I call thinking out of the box!

Edited: some rewording and breathing space

Is it that this binary does some tricks to run on Linux, Mac, Windows, FreeBSD, OpenBSD, NetBSD, BIOS or does it really work on all operating systems including something like Plan 9 or even TempleOS?

And does it run on different architectures like ARMv6?

I think it qemu's itself if run on another platform.

Native execution on x86_64 only. "Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS", where "BIOS" means that it will boot a machine and run on bare metal. On non x86_64 systems, the driver script will try to run the binary using QEMU (which has to be installed and on the path):

https://github.com/jart/cosmopolitan/blob/40291c9db386d8a952...

If i have understood how the portability works, it would be pretty straightforward to add support for another OS which uses the same ABI as the unices and has a compatible system call table - just a matter of adding a column to consts.sh (system call numbers in the nr section, constants from system headers in their appropriate sections) and adding any OS-specific system call rituals to systemfive.S.

Note that all system calls will go through the syscall interface. If your OS has some other way to make system calls, like Linux's vDSO or io_uring, that won't be used.

If your OS had a different system call table, like Windows, then you would need to write more bespoke code for it, like all the Windows wrappers:

https://github.com/jart/cosmopolitan/tree/0e85b136ae1beb0d2b...

You would then need to go through the Cosmopolitan libc and add implementations of its functions on top of your wrappers.

It's a neat coincidence that this is on at the same time as an article about CP/M, because I think there could be some overlap in the programs that would be a good fit for cosmopolitan.

Sure, there's server appliances like this, and I hope someone makes a neat services wrapper to abstract at least some platform's intricacies so you can do a "cosmo-service up redbean" on BSD/Linux/Windows.

But coming back to CP/M, there you had a lowest common denominator of terminal applications, too, but spread across different architectures. And you could still produce some quite intersting, if a bit business-like applications. These days you probably can even rely on more ANSI colors and maybe even unicode fonts (hopefull as an option, not mandatory).

The ZIP characteristics of the APE format make it even easier to distribute a whole application in a rather simple way. Yes, sure, you can do regular Unix-style servers and pipe-it-together CLI tools, but I wouldn't mind more self-contained "business" applications with a lo-fi aesthetic regarding interface and API usage. The PICO-8 of TUIs…

Author here. We're living in the most exciting time for developing terminal applications. When Microsoft unexpectedly added support for VT100 and XTERM codes to CMD.EXE it totally changed the equation and ANSI became universal for the first time. Blinkenlights is an example of a TUI application I created using Cosmopolitan and it literally works everywhere. https://justine.lol/blinkenlights/index.html You don't need curses. All that's needed is an ioctl() call which flips a bit in termios. Cosmopolitan polyfills that across operating systems. Another cool example of a demo app is this conway's game of life tui gui: https://justine.lol/apelife/index.html

Does that remove full binary compatibility for those running on older OS from Microsoft with an older cmd.exe?

Cosmopolitan supports Windows 7. The WinMain() polyfill is programmed to gracefully fall back if ANSI isn't available. So the binary still loads fine. You just need to run it in something like MinTTY instead if your TUI program is run on an old version of Windows.

I really like your view of the world, that programs should be portable, tiny, and just work. The slamming of so much functionality into a zip file is inspiring.

Would it be possible to do something similar with Free Pascal, allowing the recreation of something like Turbo Pascal, except really, really portable .com output?

Author here. Absolutely. I used to love Turbo Pascal and Delphi when I was younger. If Free Pascal uses GNU LD.BFD or LLVM LLD when it links programs, then all you'd need to do is is configure it to use cosmopolitan.a when linking system call functions like read(), write(), etc. See https://github.com/jart/cosmopolitan Another option is if Free Pascal wants to write all the system call support from scratch, then doing that now is going to be a whole lot easier since the Cosmopolitan codebase does a really good job documenting all the magic numbers you'll need. See files like https://github.com/jart/cosmopolitan/blob/master/libc/sysv/s... and https://github.com/jart/cosmopolitan/blob/master/libc/sysv/c... I've been working on a tiny C11 compiler called chibicc which has most GNU extensions and I managed to get it working as an actually portable executable with an integrated assembler: https://github.com/jart/cosmopolitan/blob/master/third_party... I also got Antirez's KILO text editor working as an APE binary. https://github.com/jart/cosmopolitan/blob/master/examples/ki... If we can build a linker too then we can get a freestanding single file toolchain + ide that's able to be a modern version of Turbo C.

I think the Zig community/BDFL might appreciate adding an APE target "os" to their buildchain (LLVM-based); that could be a match made in heaven!

Author here. I've been chatting with the Zig BDFL on Twitter recently about contributing APE support and he's been super supportive so far. It's going to be a nontrivial undertaking but would certainly be rewarding for everyone if we can make it happen.

Oh wow!!!!!!!

Very cool! Maybe I should give Zig a try too.

Also hats off to you @jart!

Is this a one time thing or are you thinking of something bigger going with this project?

If you're interested, I'd expect adding support for this to the Nim programming language would be more trivial since Nim compiles to C.

I had a quick look into doing this on your original thread[1], but got stuck pretty early on and just didn't have the time to get back into it.

If that's something you'd be willing to pursue I'd love to help any way I can. Feel free to DM me on Twitter[2] any time.

1 - https://news.ycombinator.com/item?id=25558643

2 - https://twitter.com/d0m96

Since I can't upvote your Show HN twice, I'll upvote this comment. I learned something interesting tonight from your websites. Thanks!

chibicc is a really fun idea! If I was in college, I would take a compiler course from you.

Taken from your README:

"chibicc is developed as the reference implementation for a book I'm currently writing about the C compiler and the low-level programming. The book covers the vast topic with an incremental approach; in the first chapter, readers will implement a "compiler" that accepts just a single number as a "language", which will then gain one feature at a time in each section of the book until the language that the compiler accepts matches what the C11 spec specifies. I took this incremental approach from the paper by Abdulaziz Ghuloum."

chibicc is actually written by Rui Ueyama.

Github: https://github.com/rui314/chibicc

Previous HN: https://news.ycombinator.com/item?id=24676851

I see. My mistake. So jart is trying to turn it into a APE? I learn something new with every comment.

So I would edit my comment to say: I'd love to take a compiler course from Rui, and a systems course from jart.

Edit: as pointed out by xearl, I accidentally copied the README from github user https://github.com/rui314 and attributed it incorrectly to jart.

That's what I get for browsing HN on my phone late at night.

Speaking of other languages, given that the APE is a zip container, it shouldn't be too hard to have a "starkit" like setup where the binary part is basically node/python/perl/tcl and the main script and all required libraries are in the executable.

I love what you've produced here. I noticed that pthreads seems to be fail-stubbed for the time being, i guess it's a todo with tricky semantics?

As for holding off on UI for the time being is probably the right thing to do, keeping with the minimal style planning on going for raw-x11,etc.

However, as much as I personally love plain framebuffers (late 90s democoder myself), it'd be totally non-accessible for "plain" applications and totally lacking performance compared to anything accelerated when it comes to rendering. Graphics cards are just so many times faster that it's not really even the same kind of usage (only sad part is Apple punting on OpenGL driver updates even if Intel on the low end has become so much better).

That said, having looked for a minimal UI framework and sadly they seem to be far inbetween these days (libUI is fairly compact on win/osX but required GTK on *nix but they've lost a bit of steam since 2018)

Best of luck on it all (hopefully I can snatch out some time to contribute)

So far the only platform where I've managed to get threads to work has been Windows. I came pretty close to getting clone() on Linux working a few days ago. Threads are so hard because no API was ever defined for them in original UNIX and BSD, so therefore each modern system today does them in a completely different way. Another blocker is that not all operating systems let us modify segment registers which is something that the Linux compiler assumes is available for TLS. So it'd be a miracle if it ended up being possible to make threads work portably. fork() on the other hand was easy to get working on all platforms. I love fork() and vfork().

As for GUIs I'm still open to merging raw x11 support. One tricky issue is that we'd need to define a new API that veneers framebuffers, and so far Cosmopolitan has mostly refrained from defining novel interfaces, since the focus has been twiddling with low level magic numbers in order to make the textbook interfaces we already have work everywhere!

Ah segregs is gonna be a pita indeed if it's all hard-wired to the compiler.

Considered making some way to add "portable" modules yet(generic APE code with plat-specific impls) to avoid adding stuff to the core? I know the default calling-conv is slightly different but maybe by smth like defining virtual ports? (ie "send" structs since alignment rules should be mostly the same?)

Threads are a PITA everywhere. On Linux the SYSCALL stuff complicates things because of all the register moves, the FUTEX stuff, and potential races with how threads are exited.

Now you can port Slashem to more platforms than Nethack 3.4.3 itself :D

Hey, question... Is an LD_PRELOAD version of cosmopolitan possible?

Am I the only one having issues trying to get this working? The webserver starts just fine, but once I add the index.html with zip as in the example, it stops working. This is on Mac 10.15, CentOS 8 and Ubuntu 18.04 LTS.

Centos: [centos@test ~]$ ./redbean.com -vv

  error: Uncaught SIGSEGV on test.novalocal
  ./redbean.com
  EINVAL/err=22/errno:2/GetLastError:0
  Linux test.novalocal 4.18.0-240.10.1.el8_3.x86_64 #1 SMP 
  Mon Jan 18 17:05:51 UTC 2021

On the mac: ~ my-Maccie$./redbean.com -vv Killed: 9

On Ubuntu: ubuntu@localhost:~$ ./redbean.com -vv

  error: Uncaught SIGSEGV on localhost
  ./redbean.com
  EINVAL/err=22/errno:2/GetLastError:0
  Linux localhost 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 
  28 05:20:47 UTC 2021

No, I have the same issue on Linux Mint 20.1

After adding a new file with WinRAR, I get:

EINVAL/err=87/errno:2/GetLastError:203 The parameter is incorrect.

Probably the ZIP needs to be saved in a certain format.

Author here. The zip shouldn't need to be saved in any special format. If WinRAR is posing problems then please file an issue because I'd like to fix that. That error doesn't seem like something that would indicate executable corruption. I want to learn more.

Sure thing, will provide as much info as possible.

Have to say though, using zip so you can reuse the compression is quite a masterstroke.

IDK if this is enough

https://pastebin.com/8RyPWQFf

I tried to add a file using Windows file explorer, but it just complains "The Compressed (zipped) Folder is invalid or corrupted."

This is on Windows 10 19042 build.

7-Zip doesn't even see the folders for that matter.

Author here. It works fine on RHEL5 / CentOs5 for me. I can also confirm RHEL7. Please file an issue. What will help in particular is if you can give me the faulting RIP address. That should be in the crash report or in your dmesg log.

Same here on MacOS Catalina 10.15.7. The original invocation runs fine

    $ bash -c './redbean.com -vv'
    W2021-02-26T01:50:12.665277:tool/net/redbean.c:1144:redbean:3302] setsockopt(server, IPPROTO_TCP, TCP_FASTOPEN, &yes, sizeof(yes)) → EINVAL/err=22/errno:22/GetLastError:0
    W--------------------000015:tool/net/redbean.c:1145:redbean:3302] setsockopt(server, IPPROTO_TCP, TCP_QUICKACK, &yes, sizeof(yes)) → ENOPROTOOPT/err=42/errno:42/GetLastError:0
    V--------------------000022:tool/net/redbean.c:1166:redbean:3302] 0.0.0.0:8080 listen

Visiting http://127.0.0.1:8080/ in my web browser shows a nice redbean page with lobster graphic at http://127.0.0.1:8080/tool/net/redbean.html but then following usage instructions in another shell:

    $ echo '<b>hello</b>' >index.html
    $ zip redbean.com index.html
      adding: index.html (stored 0%)
    $ ./redbean.com -vv
    Killed: 9

I blame my zip

    $ zip -v
    Copyright (c) 1990-2008 Info-ZIP - Type 'zip "-L"' for software license.
    This is Zip 3.0 (July 5th 2008), by Info-ZIP.
    Currently maintained by E. Gordon.

Addition of index.html shrinks(!) redbean.com from size 204800 to size 204348, with changes starting from char 172938. That clearly corrupts the portable executable format.

Yup, I get "Killed: 9" on macOS (Big Sur) too. I actually had some kind of problem (I think) with the original invocation:

    $ bash -c './redbean.com -vv'
    W2021-02-26T02:06:29.162953:tool/net/redbean.c:1144:redbean:83621] setsockopt(server, IPPROTO_TCP, TCP_FASTOPEN, &yes, sizeof(yes)) → EINVAL/err=22/errno:22/GetLastError:0
    W--------------------000026:tool/net/redbean.c:1145:redbean:83621] setsockopt(server, IPPROTO_TCP, TCP_QUICKACK, &yes, sizeof(yes)) → ENOPROTOOPT/err=42/errno:42/GetLastError:0
    error:./redbean.com: check failed: 0xffffffffffffffff != 0xffffffffffffffff (48)
    6fffffffff70 0000004034cf UNKNOWN
    6fffffffff80 000000403225 UNKNOWN
    6fffffffffc0 0000004027f3 UNKNOWN
    6fffffffffe0 000000402a21 UNKNOWN
    7ffeefbff870 00000040227b UNKNOWN

IIRC, from vague memories of things flying past, Big Sur won't let a modified binary run once it's been checked by Gatekeeper. Which means the first run gets checked and notarised but when you add the `index.html`, the on-disk binary has changed and Gatekeeper won't allow it to run - I guess it's to prevent malicious code modifications, etc.

Oh wow that's good to know. Maybe you have to do something like `cat foo.com >foo2.com` after you've modified it with the zip tool.

Yeah, but this isn't Big Sur, it's Catalina. But maybe it's the same protection there as well?

The binary didn't work on Ubuntu 18.04 nor CentOS8 either, but I'll report a issue later on today.

Thank goodness AAPL is here to protect us from doing cool stuff.

Its good from a security perspective, but also its a nightmare to know that the mothership knows everything you run on your machine.

This will also prevent distribution from outside of their appstore giving it will prevent updates.

So as anything Apple, the world security here is being used as code for personal machines remotely controlled by the mothership.

You trade privacy and freedoms for a little convenience.

> This will also prevent distribution from outside of their appstore giving it will prevent updates.

I think that if an app is correctly codesigned, Gatekeeper has no issue with it. Also if it's not correctly codesigned but you've ticked "allow unsigned binaries", you can still run them.

> the mothership knows everything you run on your machine

I can't remember the details from the last time this came up but IIRC it only sends a hash and possibly even then only the first time you launch it. Either way, they're not going to block out-of-appstore distribution except if you're a known malicious actor.

It looks like you can get around this by adding Terminal.app as a "developer tool" (i.e. it can create processes without hitting Gatekeeper), maybe that works? (once I did that and relaunched terminal, I can run cosmopolitan binaries).

https://docwhat.org/upgrading-to-catalina

What happens if you switch off Gatekeeper?

`sudo spctl --master-disable`

Yeah another killed: 9 on Mac BigSur. Could you export the tool in a way that we don't have to zip new files into it? Instead we would have to gcc the whole thing once and be done with it.

Someone already reported the same https://github.com/jart/cosmopolitan/issues/56

On Mac Big Sur it doesn't seem to work either:

    ▶ zsh -c './redbean.com -vv' 
    W2021-02-26T12:12:24.019387:tool/net/redbean.c:1144:redbean:56779] setsockopt(server, IPPROTO_TCP, TCP_FASTOPEN, &yes, sizeof(yes)) → EINVAL/err=22/errno:22/GetLastError:0

W--------------------000028:tool/net/redbean.c:1145:redbean:56779] setsockopt(server, IPPROTO_TCP, TCP_QUICKACK, &yes, sizeof(yes)) → ENOPROTOOPT/err=42/errno:42/GetLastError:0 V--------------------000023:tool/net/redbean.c:1166:redbean:56779] 0.0.0.0:8080 listen ^CV2021-02-26T12:12:27.012972:tool/net/redbean.c:1183:redbean:56779] 0.0.0.0:8080 terminated

EDIT: I am an idiot: it's working just fine... the messages looked like an error to me, but visiting https://localhost:8080/ does show a nice HTML page!

Disable Gatekeeper:

  sudo spctl --master-disable

No luck for me on win10. I can open the archive but get errors trying to add to it. 7zip and win explorer zip both fail with either unsupported or corrupted archive errors.

Since it's a .com file, after downloading, win 10 also claims 'this dangerous file comes from the internet and is blocked' which you have to check a box to unblock it. I did, but still get the errors trying to add to it.

Fixed for me (macOS) in this[1] commit.

[1] https://github.com/jart/cosmopolitan/commit/218ef491476f15ab...

If you want to learn more how these things work I'd highly suggest going through the PoC||GTFO archive (https://github.com/angea/pocorgtfo/blob/master/README.md) and check out entries by Ange Albertini or entries named like "This ZIP is also a PDF".

I came here for this. E.g. https://www.alchemistowl.org/pocorgtfo/pocorgtfo19.pdf

"This file, pocorgtfo19.pdf, is valid as a PDF document, a ZIP archive, and a HTML page. It is also available as a Windows PE executable, a PNG image and an MP4 video, all of which have the same MD5 as this PDF."

Zero/near-zero dependencies, minimal configuration, portable, and trivial to run. Awesome.

Even if Redbean here doesn't end up taking over the world, I hope that more ecosystems and toolchains sit up and take notice. We need more tools that look like this.

The amount of work that went to this is insane. She made a (full ?) implementation of libc: https://github.com/jart/cosmopolitan/tree/master/libc

Right? It's amazing to see what some people can accomplish with the same 24 hours a day we all get.

OK I see we are going crazy, me too. Thanks @jart.

I have been thinking about portable web apps with embedded SQLite for some time. I do not have the technical chops at C level to pull this off. I am really inspired by this project. I hope Redbean leads a way to distribute self-hosted apps in today's era of the cloud.

I still use the classic version of Tiddlywiki that is a single self-contained self-editing html file, and I "app-ify" it on Windows by changing the extension to *.hta.

But having the similar solution with the SQLite, would be a perfect combination.

I feel like I’ve learned more from your website and your source code in a few hours than I otherwise would have from years of study.

You have a real talent for distilling the complex into something simple. It’s apparent in the purpose of your programs, in your writing, and in your source code. Thank you so much!

Basics of computing architecture that I didn’t especially think were too complex for me to learn, but we’re too abstracted for me to become interested in, are laid bare in each one of these programs and their source. This is all incredibly exciting.

Just make sure you don't have another server running on :8080.

I had syncthing running and kept getting some error whenever I tried to run redbean. The error didn't make much sense, but eventually I realised that this was the casue of the error.

I'm really impressed how portable this is. The only improvement I can see is if it auto-opened your browser to :8080. That way it would be easy to distribute the binary and have people run your application and just interact with it through their browser. No need to ship electron then!

Let's say we are able to do this. Would we have any security concerns to be addressed?

How different would it be from opening a random site in your web browser?

There is the possibility of other programs making HTTP requests so you shouldn't assume trust on any request automatically. What you can do is give the browser a cookie once when your app starts which you then check on every request.

Oh and listen on 127.0.0.1, not 0.0.0.0. That way your app is not needlessly exposed to the network.

I think you could trivially issue a system() call, you'd just have to use the corresponding command to open the default browser for each particular system. (Linux: xdg-open, MacOs: open, Windows: start)

I am floored. Not only is this just breathtaking sorcery, I also needed this exact thing for a project. I don’t understand probably 70% of the description of this software, but I understand what it does, and I’m more excited to carefully study and use this than I have been about anything in years.

TLDR: This is a packed file format that can look to the OS as being executable (for Windows/Linux/OSX). The code is x86 and so offers native performance on those processors. On other platforms, an x86 emulator is built in so can't offer native execution speeds.

While neat, its not the "best of all worlds" due to the lack of native code on anything other than x86/x64. Also, claiming "bare metal" is supported is a stretch as you are limited to having no I/O of any form (as there is no platform code).

No I/O eh?

I'm sure the FP zealots will love it :-)

I'm just one woman. What Cosmopolitan does so far, it does really well. It was only as recently as a few days ago that I got mmap() and malloc() polyfilled on bare metal. It has serial uart i/o. It's going to have e1000/virtio sockets soon. You can help me will that future into existence. I need people who know o/s dev and can help me write code that does things like correctly set up pic controller.

Sorry, wasn't meant as a criticism, just as an explanation because people were assuming magic...

Taking on bare-metal is a significant job much greater than your original task and you may be better targeting something like buildroot instead, after all what is the Linux kernel but a hardware abstraction layer.

How hard could stdio/sockets on metal possibly be? So far bare metal has been a walk in the park. I love the fact that PML4T lets me do things like move memory without copying it. I want to be able to have that power without schlepping in the entire Linux world. I believe it should be possible for programs to be able to boot on metal as just programs which are tiny and auditable. Especially considering everything runs on hypervisors these days, which are in some respects the true operating system. Ring 0 is the new Ring 3. Who among you is willing to accept being pushed to the outer bounds of computing? If MINIX gets to run in Ring -3 then it should at least be easy for our programs to run in Ring0.

As someone who knows programming in higher level languages well but has no experience with low level, bare metal, how can I get started with this stuff you're working with?

> So far bare metal has been a walk in the park. I love the fact that PML4T lets me do things like move memory without copying it.

> I believe it should be possible for programs to be able to boot on metal as just programs which are tiny and auditable.

I would love to be able to learn more about this. Maybe even contribute to your project :) perhaps a good way to start would be to address some small tasks you might have laying around?!

And I would love to encourage you to do so! If you're working for Cosmopolitan then I'll be working hard to support you. There's a long history of depth to metal. Many people start off learning it by having fun with the old skool ibm pc boot process. Here's a gentle introduction: https://justine.lol/blinkenlights/realmode.html and https://github.com/jart/sectorlisp and http://reddit.com/r/osdev

Those concepts are implemented in the Cosmopolitan codebase here: https://github.com/jart/cosmopolitan/blob/master/ape/ape.S

That file runs a few hundred lines of old school assembly in order to bring us into the modern era. Towards the end of the file you'll notice it starts calling functions that are written in C which configure memory: https://github.com/jart/cosmopolitan/blob/master/libc/runtim...

After it configures memory, the ape.S file is able to call _start(). It also sets a bit in __hostos so that functions like write() know to use the serial port instead of issuing system calls: https://github.com/jart/cosmopolitan/blob/master/libc/calls/... That's the simplest possible example of a driver with spin locks which uses the x86 IN/OUT instructions.

If you want to take a dive off the deep end right now with Cosmopolitan on metal -- simulated -- then try the following build commands on your Linux terminal:

    git clone https://github.com/jart/cosmopolitan
    cd cosmopolitan
    make -j8 o//examples/hello2.com
    qemu-system-x86_64 -m 16 -nographic -fda o//examples/hello2.com
    sed -i -e 's/USE_SYMBOL_HACK 0/USE_SYMBOL_HACK 1/' ape/ape.S
    make -j8 o//examples/hello2.com o//tool/build/blinkenlights.com
    o//tool/build/blinkenlights.com -rt o//examples/hello2.com

Its not the difficulty, but the scale.

Working on one bare metal system is relatively easy as long as you're willing to dive into drivers. Getting 1001 I2C drivers or maybe writing a drivers for tens of different types of flash devices is a different matter.

Your project will quickly become overrun with drivers for literally tens of thousands of devices... then comes maintenance.

Look at the Linux source tree... Good drivers are not trivial either. Simple ones are of course.

Basically, its a time-sink and it doesn't become useful for claiming "portability" until you're approaching the scale of something like Linux.

You're confusing portability with requirements. I'm aware that a long tail exists. Linux is a big tent for that sort of thing. I think that's great. I also want the freedom to live outside that tent during the times when all I need are the other 85% of use cases, which as I've shown, can be easily achieved by just one person. I don't believe that just because I ship actually portable executables that are able to do e1000/virtio that every hardware manufacturer is going to be kicking down my door eager to get support for their silicon merged. Linux is already doing a great job at that and I think Cosmopolitan is complementary.

Can someone marry this with QuickJS and make a micro node competitor with a web server, 0 deps and no install?

Wonder how far something like that could go, could be really interesting for replacing bash for me with something easier to write like JS.

This is so powerful that it can be used as a "weapon": it's the perfect lib for making malware. I think this is a compliment!

scary scary scary

She started her career with a very successful spam tool. She was 14 at the time.

I definitely was not coding when I was 14. Dang.

Can someone explain this to me like I'm an 8 year old?

Can someone explain this to me like I'm a 31 year old?

I've been programming since before you were born... and this is the most impressive hacking feat I've ever seen.

A single executable, that doubles as a valid ZIP file, that you can put a complete website into, also doubles as a valid executable on Windows, Mac, Linux.

The build infrastructure is a complete reworking of C to make it far more portable than even K&R thought possible.

I probably wouldn't have believed it possible, but here it is!

Thanks!

Multi OS/arch executables exists for decades. They were called fat-binaries[0]. Old trick is new again.

[0]: https://en.wikipedia.org/wiki/Fat_binary

https://en.wikipedia.org/wiki/Universal_binary

But compared to what jart has done, those were effectively cheating. ;) They just smooshed multiple binaries into a single file.

On most computer systems these days can select a set of files and folders and tell the computer to "Send to compressed file", the files that come out of that process usually end in ".zip", and are thus known as "zip files"

This is a zip file, like any other... you can add and remove things from it. However, this ZIP file has a superpower, it can display the contents on the web... it has a web server built into it.

You can take this file, run it on a Windows, Linux, Macintosh, and it will work. You don't need separate versions for each different system... this just works on all three systems, unlike almost every program ever written.

This is one of the most impressive feats of programming things I've seen in my 40+ years of programming. The web serving is clever, but the superpower that it can run on anything really took a huge amount of work, which the author built into a tool called "ape", and has shared with the world, so other people can use it, and help her make it better.

So... will a new browser need to be created in order for this tech to go mainstream? In the case of building a web app does this eliminate the need for a server?

Someone please be kind and explain to me how this works? Don't different executable formats need different headers?

Don't ELF need "0x7F ELF" header? Don't Windows executable need "MZ" header? How is it fulfilling both requirements simultaneously?

The OP explains https://justine.lol/ape.html

I will say, I WOULD absolutely love it, if it weren't for the little problem...that...I can't "just add to the zip"...because for whatever reason, it ruins the special setup... on windows, 7zip and windows explorer's builtin zip stuff refuse to modify it, as if it was corrupted (understandable), but explorer can view it, on linux, zip (apparently info-zip 3.0 according to the version) can add to it just fine, but ruins the special properties, on linux it just spawns the debug gdb with "no symbol table loaded" and no register or assembly data, and on windows it's either wrong format, or `check failed: 0x1 == 0x0 (32)` so also broken... I definitely love the idea of this, and the APE in general, and it works as-is, but modification seems impossible to me...changing to .zip doesn't change the breakage

anyone else had luck with adding stuff to it? is it just me? or is it like "it works with openbsd zip command only"?

I looked into building SBCL using chibicc last weekend in the hope that one could build a truly portable lisp runtime using the ape toolchain (Yes, yes, I know you couldn't possibly pick two more different approaches to software portability). Turns out the GNUC extensions for __asm__ are the primary stumbling block (though I'm sure there are others). tcc supports the GNUC extensions needed, and chibicc supports the thread local storage needed for dynamic variables. Looking forward to future developments!

You need __asm__ in chibicc for sbcl? I have good news for you. I did exactly that in cosmopolitan's vendored fork of chibicc here: https://github.com/jart/cosmopolitan/blob/master/third_party... I went on a coding rampage back in December, adding to chibicc pretty much every GNU extension under the moon: https://github.com/jart/cosmopolitan/blob/master/third_party... It should be a perfect fit for use cases needed by Cosmopolitan and SBCL. Right now it's unlikely to get merged back upstream because Rui is still focusing on using Chibicc for his book, and that requires being more conservative about feature inclusion right now.

I would highly recommend using chibicc. Bellard's work on tcc was fantastic when it came out. However it didn't age well. Its GNU extension support is roughly equivalent to GCC 2.x. The x86_64 support that got bolted on later isn't very good. It was much more elegant back when it was only doing i386. Hackability was also laid low by merging a lot of external contributions. The TCC design, while amazingly fast, also carries the tradeoff of making things like inline assembly hacks really hard because it generates the x86 binary content directly, rather than going through the intermediate step of generating an assembly file and running it through a proper assembler -- which chibicc now has!

I'm actually happy to merge your GNU-style inline assembly support and other GNU C extensions to chibicc. But for the purpose of the project, I need to rewrite your patches in the incremental manner, so it's not something I can do right now. I'm currently busy working on my linker project. Once it settles down, I'll take a closer look at your fork.

Thanks for clarifying Rui. I'm happy to do the bulk of that effort for you. As soon as you're ready, just tell me which things you want to see upstreamed, and how fine/coarse you want the granularity to be, and I'll send you pull requests!

That really depends on how I organize chapters of the book, and it will need trial and error, just like I'm continue rewriting book chapters as well as the compiler's commit history. So I think I have to do this myself.

Windows 10 (10.0.18363 Build 18363). Just downloaded and executed. No zip modification at all.

  W2021-02-26T05:00:59.959364:tool/net/redbean.c:1141:redbean-2021-02-25:49336] setsockopt(server, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes)) → ENOPROTOOPT/err=10042/errno:10042/GetLastError:0
  W--------------------000674:tool/net/redbean.c:1145:redbean-2021-02-25:49336] setsockopt(server, IPPROTO_TCP, TCP_QUICKACK, &yes, sizeof(yes)) → ENOPROTOOPT/err=10042/errno:10042/GetLastError:0

Its amazing if works though, i can put my static site in an exe. That is so cool.

EDIT: Runs on without modifying even with this message.

EDIT: Cannot modify zip on windows. "File not found or no read permission" using explorer. With 7-zip "Operation is not supported."

EDIT: Used WSL to update zip which then results in SIGSEGV

>the sources that went into creating it are nontrivial and only recommended reading if you really love printing lemurs in terminals.

this is delightful

Everybody is hyped up here but can someone explain to me what are the implications for the future of programming?

Will I be able to write apps in a single language and have then run truly native on any OS or what else will we be able to do?

This is something like a "fat binary" with some magic in the header that allows it to run out of the box on Windows and several different UNIXes, a bit like those sneaky shell scripts that run on several platforms[0]. It still needs to include some native code for each of the platforms inside the compiled binary, so it's perhaps better to see it as a slick packaging/distribution solution for C programmers than a total game changer for developers everywhere.

The more interesting thing I think is the mapping of C library functionality between Linux, BSD and Windows. There have been other attempts to do this, but they never seem to get much traction. I think it's because C programmers tend to get wedded to their OS of choice and then invest time in improving that platform's C libraries. The problem with trying to remain portable with everything is that you tend to be stuck working with the lowest common denominator functionality, which can be frustrating when doing OS-specific stuff would open up so many more options.

All that said, the cleverness of this project is inspirational in a way that might encourage other programmers to also focus more on building against simple functionality that is truly portable. It shows that you don't need all the bells and whistles to build cool stuff. That feels aligned with early UNIX philosophy and (more broadly) the hacker ethic, which is why it's neat to see, even if never gets adopted as a mainstream thing.

[0] https://github.com/llamasoft/polyshell

Appreciate the response. So it's a bigger win for C devs specifically. I am a C# dev so I am trying to understand what opportunities this project unlocks.

Why you have pigeonholed yourself? "X Dev" is so cliche. You can compile C# to x86 machine code - this project has less to do with C and more to do with PE format and syscall tables. It means that it's language agnostic but not standard library agnostic.

Completely agree. The program is written in C# -- you aren't! :-)

Nice job, Justine. This level of excitement reminds me of the early days of Linux. I think it has the potential to go far.

Author here. Thanks! I was deeply inspired when I read the source code to early versions of Linux and UNIX. The dream of that simple elegant power is something I'm hoping to restore for a new generation. All I'm doing to achieve that is simply standing on the shoulders of the giants who came before me.

Have you considered trying to make system calls inline to avoid the call overhead? Or already thought about it?

I've been banging my head against the wall of trying to make a micro KVM guest and the glibc startup code uses inline SYSCALL everywhere, I assume to make things faster. It does seem to call CPUID a lot too, but I guess it's not as expensive as having to make full-blown system calls.

Cosmopolitan defines linkable symbols for all the __NR_syscall constants so you can absolutely use inlining if you like to live dangerously. For example: https://github.com/jart/cosmopolitan/blob/91f4167a45f811fde6... There really isn't any performance advantage to doing this, because the SYSCALL instruction itself has something like a 2000+ cycle cost due to all the copying that needs to happen on the kernel side along with things like spectre mitigations which have made is slower. So the 8 cycle cost of having a normal read() function wrapper is the wrong thing to be focusing on. Cosmopolitan aims to address the performance cost of SYSCALL by making it possible to run your binary on bare metal, where your program becomes a kernel, and therefore needn't pay any context switching costs at all.

And it's even more overhead when exiting a VM, then?

I can't believe I'm ASKING for a blog post.

Can you blog a bit of the career/cognitive/skill evolutionary history behind this? Did the cosmo stuff come out of this or something else?

Learned this in X level school, found out about Y, read about Y, learned this, got this job doing this, learned this from Z, got experience of AA... etc.

Of course, assuming this exists, and you exist. And blogs "exist".

Author here. When I first discovered the idea for Actually Portable Executable I thought I was going mad like Terry Davis. I couldn't even believe that it was possible until I actually did it. It cost me my job but I couldn't be happier with the result. This project is going to make problems disappear for so many developers. It's the sort of tool people can get excited about. I'm so happy that my prior career success and top-notch training have granted me the privilege of bringing something this cool to the world. Enjoy!

In all seriousness: whether it's exciting or not, this is something that should be getting federal funding. It really shouldn't be difficult for large, government-backed organizations to justify their interest. I don't have a lead for you (otherwise, I'd have invoked it by now to work on my own project[1], which has something in common through the use of polyglot files to achieve a similar effect[2], while mostly otherwise taking a different approach), but since you have a high profile, casting a wide net to attract the attention of someone able to secure the funding is something to consider.

1. <https://wiki.triplescripts.org/wiki/The_why_and_how_of_tripl...>

2. E.g., <https://www.youtube.com/watch?v=TUpd70Mu0Ek>

.apk files and .ipa files are secretly zip files too. With the addition of a few manifest files and other crufty things embedded in the zip, I bet you could add Android and iOS to the list of supported platforms too.

I've recently succeeded in building working .apk files byte-by-byte completely from scratch so I could maybe help some. Open an issue on https://github.com/akavel/hellomello if interested in some talking/advice and/or watch my talk on NimConf 2020 for an overview (I don't have a link handy as I'm on mobile).

Jars, rpms, whls, Microsoft Office, probably a bunch of other stuff I don’t even know about are actually zip files.

.epubs too!

The magic numbers file ends with a comment linking to a youtube video of numbers station playing very disconcerting and dissonant music, so we can add art/comedy to the list of features (which was already obvious!)

I’m not an expert at C code but it’s really nice to read redbean.c - lots of relatively small functions with great names. Good example to follow.

> That performance is thanks to zip and gzip using the same compression format, which enables kernelspace copies

Can someone elaborate on the relationship between gzip and kernel space copies?

Normally when a browser asks for Content-Encoding: gzip, the http server needs to schlep the file into memory, compress it, and only then is it able to send the output to the client. Since redbean reads files from a zip archive, that file is already compressed. So what it does is it uses mmap() to load the zip executable in memory. Therefore instead of read()+write() it can just call writev() on the mapped memory area. That enables us to skip the extra copy. The kernel can just copy the gzipped file contents directly to the network wire without having to pay any userspace context switching penalties.

"I figured out how to microwave water, to make gold."

Good lord this is cool

This is bonkers amazing!

I love the reference to the Swedish Rhapsody numbers station at the end of consts.sh

This is awesome, I've wanted a way to package a webapp for distribution locally and I think this + sqlite would be a perfect combination.

This is amazing! There are a lot of low level things that I don't really grasp and, while going through the APE description, I saw this: "The other tradeoff is the GCC Runtime Exception forbids code morphing, but I already took care of that for you, by rewriting the GNU runtimes".

Can somebody, please, explain if allowing code morphing will increase security risks?

No relation to http://redbeanphp.com/ I guess :D

I think that's still worth mentioning considering they're roughly in the same sphere of interest, to avoid the confusion.

And now we have a portable format to embed cross platform an entire interractive book with zero knowledge needed from the reader.

> exes still end up being roughly 100x smaller than Go Hello World

> Is the Cosmopolitan Runtime lean and mean like Go's 2mb Hello World executables?

:-)

Odd, I'm getting $ ./printimage.com image.jpg

run-detectors: unable to find an interpreter for ./printimage.com

Author here. That error happens if you have binfmt_misc enabled. The solution is simple:

    sudo sh -c "echo ':APE:M::MZqFpD::/bin/sh:' >/proc/sys/fs/binfmt_misc/register"

I intend to upstream a patch with the Linux kernel so it can do this automatically. If any kernel devs are reading, please email me!

Does it work on cheap Xen vps? Because "/proc/sys/fs/binfmt_misc/register" looks absent from mine.

That's strange. I'd recommend filing an issue. Any details on how I can reproduce the error will be helpful, even if I have to temporarily rent one of those things. I'm on a mission to make binaries portable.

FWIW, it works on QubesOS, which is Xen based. Though it works only in my Fedora and Debian VMs, but not on my Archlinux-based one.

I think that's because I have WINE installed in the Archlinux VMs, since that's what gets started when I run ./redbean.com. But even after I remove WINE, it still doesn't work, I probably have to restart (i.e. remove WINE from the TemplateVM, not the AppVM) for it to work.

In any case, incredible work, I am in awe!

Instead of sudo sh -c to sudo the redirects, consider echo 'abcde' |sudo tee /proc/bla. Less nesting :)

Do you mean less syntactical nesting? The pipe to sudo tee creates a new child shell, just as sudo sh -c does.

Yes.

Awesome! You could even use a zip file as a container for the assets you want to serve! see https://gildas-lormeau.github.io/ (the page is a self-extracted zip file). And here's a demo showing how to update an image: https://www.youtube.com/watch?v=RJWhDmYhnX4

Why does the magic numbers list have a XENIX column?

Author here. The name was chosen mostly out of playfulness to describe Windows since I seem to recall XENIX being the earliest example of Microsoft dipping their toes in the water with UNIX support.

Back in the day ('80s) worked at Logica with colleagues who did Xenix support!

Cute idea. Sounds like magic.

Built with actual magic: https://storage.googleapis.com/justine/ape.html

From the same person I think

Author mentions that the APE’s can be booted from the BIOS. Would this imply you could perhaps implement a lambda-like cloud service that could spawn/start the APE on demand? Edit, I’d guess you’d want to/have to, also have a tcp/ip stack in the APE as well (but perhaps just going for containerization of the APE is the better play and not have to worry about any of that..)

The trademark collision is going to test the joke of Red Bean Software:

> We're software producers of a different kind. Rather than market software the usual way, we openly deny its existence, and challenge our customers to do the same.

https://red-bean.com/

It is all pretty good, but why there is update of "static bool" variables from signal handlers? In old-style C it must be "volatile int" (and only int), in new-style moder C it must be atomic-anything (bool is ok). But access to plain "static bool" from signal handlers & main thread is race and UB.

Would GUI programs be possible with this? I don't mean some cross platform framework monstrosity.

But if some least common denominator was found to be able to create window, add some input fields and buttons using the native UI elements of the OS?

Does the Win32 and Cocoa APIs for example, also have "magic numbers" that are kept stable?

> Would GUI programs be possible with this?

This is a cross platform web server, not a cross platform web browser.

Sorry, I forgot that this post was about the web server and not about αcτµαlly pδrταblε εxεcµταblε

I can't get it to run on Ubuntu. "run-detectors: unable to find an interpreter for ./redbean.com"

Presumably X86_64 only?

Not sure the is a working example. But they talk of embeding qemu to create cross archetecture executables. But the Window's and actual compiled code would still be x86-64.

She says somewhere in her website that arm/apple silicon support is really easy to add, with no significant size or performance penalty, the same way she added qemu support.

This indeed is a beautiful thinking about things. Truly worthy of HN front page.

I keep getting segfaults in running the executable, only after zip ...

I know this is x86_64, and have an am64 system. Does this only run on intel and arm architectures?

Does this use the same exploit in memory that spectre used for intel processors?

I'm unaware of an architecture named am64, so I assume you mean amd64, in which case that _is_ x86_64. Same ISA, different name.

If you're getting `check failed: 0xffffffffffffffff != 0xffffffffffffffff (98)` make sure you're able to bind the default port (8080). You can run the symbolized version to see where the check occurs.

The APE shit is really cool.

The webserver is novel but I am just wondering, how would it compare to something like Apache/ISS in terms on security/config/etc, or is this mostly just to move small sites around?

I was literally thinking about how cool this would be if it existed like a few days ago!!! Down to the freaky portable executable!

Congrats on reading my mind and making what was supposed to be impossible?

Does it work with this level of bare metal

https://news.ycombinator.com/item?id=26271356

I tried to run it on an old 32bit pc with Linux. But it complained about not knowing the format of the executable. But it would be awesome if I could get it to work somehow.

Build a qemu to run a 64bit guest.

Alas, I wanted to try this out, but the format shenanigans triggered the Sophos "machine-learning"-based threat detection on my Windows laptop in mid-download.

Can it be extended in a way similar to CGI back in the day? I would just want to compile it with the CGI program rather than expect it to dynamically load.

You should file a feature request! We could crowdsource some ideas about the best way to do that. For example if you read the APE blog post, we've already got a JavaScript interpreter (duktape) checked-in to the Cosmopolitan codebase. I could easily link that into redbean and define possibly a nodejs-like API.

Out of curiosity, are there any general benchmarks available as to how this stacks up to something like nginx when it comes to static page serving?

As amazing as this is unfortunately it doesn't work for me and I am on Linux.

I get bash: ./redbean.com: No such file or directory

after chmod-ing.

Maybe I am dumb but yeah.

Is your system 32 bit by chance?

No, not really. It's x64. I really wanted to give this a try.

Did you make it executable with chmod +x ?

Yes. That's the funny part. It doesn't work despite that. uname -r gives 5.7.7-amd64-desktop.

I get the same :( I'm on Archlinux.

When I downloaded the file, it also had the date in the name. You have to accommodate that change in the instructions. Also be sure to run bash -c './redbean.com -vv' (also with the name as downloaded).

Am I understanding this correctly that you can also zip up any binary which can further be served by the webserver?

Is it a "fat" binary?

AFAIK a fat binary is just a binary with all dependencies included. It would still be compiled for a specific platform. The magic part about this is that it's a single file compiled once that can be run on most platforms. And it also adheres to the format of a zip file, so you can add, modify and remove assets as you please inside the actual file, post compilation.

No, fat binaries are not binaries with all dependencies included. Fat binaries are executable files that contain equivalent compiled code for multiple architectures. Some also facilitate running binaries across different operating systems. More: https://en.wikipedia.org/wiki/Fat_binary

The idea also reminds me of (executable) self-extracting ZIP archives that were common once upon a time.

Thanks! Seems like I've misunderstood the concept a bit. My apologies.

error:./printimage.com: check failed: 0x0 != 0x0 (0)

6ffffffffe30 0000004025db UNKNOWN

6ffffffffe40 0000004023a7 UNKNOWN

6fffffffffb0 000000401862 UNKNOWN

6ffffffffff0 000000401ed1 UNKNOWN

Run on ubuntu 14? There's nothing running on :8080. Not sure what I'm going wrong.

Upd: works. printimage didn't like the jpeg I tried testing with the first time.

This is the coolest thing I’ve seen on HN in a long time. Nice work

rather annoyingly, when seeking a PHP implementation to use with this, there is in fact a project called RedBeanPHP, which seems completely unrelated

crashes on Ubuntu 20 after zipping, before zipping works fine. Really cool idea

This is brilliant.

this is outstanding! speechless.

One small complaint I have is that there is already a popular project called RedBean: https://www.redbeanphp.com/

So it's a tiny virtual machine. Intriguing, but not really shocking.

What is your definition of a virtual machine and what do you think is happening here?

"no wireless. less space than a nomad"

>So it's a tiny virtual machine. Intriguing, but not really shocking.

Nope... it's native code, that runs on multiple platforms, all in one small executable.

How? Lots of very clever hacking, and trimming all the accumulated dreck out of the normal C runtime, while making it work across all the supported platforms. (Polyfill is the term she uses)

This is the most amazing thing I've seen in a decade or so. It is right up there with GIT in terms of the possibility space it opens up.

Caddy is a single file web server as well :) https://caddyserver.com/

Admittedly they solve different usecases, where Redbean seems to want to serve a whole static site as a single file, which isn't currently possible with Caddy, (but you can run `caddy file-server` to serve the current directory as a static site), but it may become possible in the future with Golang's new embed package https://golang.org/pkg/embed/

That's actually a pretty interesting idea. I wonder if we could make a lite version of Caddy bundled with your website embedded...no need to mess with file systems or permissions or devices and folders to run a static site. Might need some massaging to become a truly pleasant workflow though.

Yeah - I could see it being part of xcaddy tooling to package up the static assets in the binary :)

The go std library offers the embed package. You can just embed your files and use the std lib's webserver and generate a single file binary. No need for Caddy.

All this would be roughly 15 lines of code.

People use Caddy for automatic HTTPS, which the std lib does not give you.

Obviously you don't need to know this but others might not be aware that certmagic is a great middle ground between Caddy and rolling your own ACME.

Caddy is cool! Redbean is magic :p

Thanks for sharing.

Caddy is cool but it has a separate 33mb executable for each operating system. Redbean is a 128kb executable that runs on all platforms so it has fewer moving parts and is 1584x tinier. All the convenience of being able to "just add your assets to the zip executable" wouldn't have been feasible if we needed to repeat that process n times for each operating system.

Yes of course - like I said, they solve different usecases. Caddy has way, way more features overall to make up for the bigger binary.

Is that smile at the end of your first sentence meant to be condescending?

You are affiliated with Caddy and ought to mention that when you promote your own software. https://news.ycombinator.com/item?id=16687686

This is not a competition. Caddy is not even in the same realm as Redbean.

Redbean is a multi-platform binary that doubles as a ZIP. It's roughly 1500 times smaller than Caddy too.

> different usecases

Right. :)

> Is that smile at the end of your first sentence meant to be condescending?

What? Of course not! You're reading way too much into that. It's simply enthusiasm. I read about Redbean and it being a single file webserver and the first thing to come to mind is Caddy, as "prior art".

Like I said, different usecases; Caddy is a general purpose webserver which has automatic certificate management, H2/H3 support, reverse proxy, file server, a strong plugin system, the list goes on. Of course it'll be bigger in size. Redbean is super cool and does have the advantage in file size and portability, clearly, but it is essentially a single-purpose tool. And there's nothing wrong with that.

Ok well this doesn’t work on Mac, so much for not needing to cross-compile on every architecture. It’s because of the OSX Gatekeeper but if majority of people need to self-sign or otherwise compile this binary I’m not sure if it is that portable be it if it’s only a Mac thing. This dumps the zip content directly on the wire skipping an extra kernel write. There have been attempts before to run the same binary on multiple architectures by inserting magic headers without having to recompile. There are shell/batch versions of the same thing [0]. The issue is such projects usually never get much traction since every programmer is wedded to their choice of OS and typically relies on APIs specific to each system. There are also “fat binaries” which accomplish the same thing if one truly wanted a cross platform single executable. There isn’t a huge need for them currently and not just because they’re “fat” (something this approach seems to tackle primarily) so I expect for the same reasons there won’t be many needs for this approach either.

I see a lot of people making the comparison between this and the OG Linux tools. I absolutely don’t think this qualifies as one, most old Linux tools are irreplaceable or at least were when they first came out and there were no alternatives. Serving a bundled zip file on a web server across many different platforms is not something people are dying to do. You can run a web server like nginx or Apache to index your files or if you really hate those why not a single line command which is supported on almost any platform:

  python -m http.server 8080

[0] https://github.com/llamasoft/polyshell

Youre really missing the point here.

This zip file is an executable webserver, you just add files to it and it will serve them when you run it.

If you can't understand what is cool about that maybe just don't speak.

Reply on news.ycombinator.com