Patching GCC to build Actually Portable Executables
ahgamut.github.io
ahgamut.github.io
With my gcc patch, you can now build software like vim, emacs, ninja, bash, git, gcc etc with Cosmopolitan Libc, via their usual autotools/cmake-style build system. The built executables should run on Linux, FreeBSD, MacOS, OpenBSD, NetBSD, and also Windows (although I haven't tested Windows yet.)
Here's a list of software I got to build with this technique: https://github.com/ahgamut/superconfigure
The superconfigure script is just a wrapper around the usual configure script used to build your software, supplying flags like --enable-static.
If you want to build gcc using Cosmopolitan Libc -- try out this repo: https://github.com/ahgamut/musl-cross-make/tree/gccbuild
In the meantime, the Cosmopolitan Libc monorepo contains binaries of gcc-11 and binutils that have been built with my patch, you can use those for now :)
This gcc patch makes such build scripts simpler, because you will need to change less of your code -- let me know how it works!
The default build parameters in busybox make a lot of OS-related assumptions, so I'd recommend you use make menuconfig instead of make depconfig. in the TUI, you can disable all console and networking utilities, linux modprobe utilities, some linux system utilities, free, uptime, etc. Start with a minimal config and then slowly add things.
In terms of source code changes: modify `u_signal_names.c` to use a switch statement instead of the amazing "use SIGHUP as an array index" method it follows, use int32_t instead of the "smallint" type that busybox seems to prefer, and IIRC there's an enum in there somewhere that should be rewritten as a #define.
That should get you pretty close to a successful build.
[unconstructive complaint about lobste.rs moderation skipped]
I hope HN treats you better.
Lobsters has had serious problems with users who try to exploit the site for self-promotion, and it's more serious for us than HN when a few votes is enough to move something to our homepage. As part of addressing it, we have a bright-line rule prohibiting new users from submitting stories from new, unseen domains in their first 70 days on the site.
Five hours after jart invited woodrush, at 2022-01-12 02:26, they tried to violate the rule and got this error message:
> woodrush.github.io is an unseen domain from a new user. We restrict this to discourage self-promotion and give you time to learn about topicality. Skirting this with a URL shortener or tweet or something will probably earn a ban.
At 2022-01-12 02:40, jart submitted the exact same link to break the restriction, and woodrush submitted several posts from that domain. After that, jart started submitting stories from all of her invitees to circumvent this rule, which anyone can verify by comparing timestamps on her submitted stories at https://lobste.rs/newest/jart to creation dates for her invitees at https://lobste.rs/u#jart.
On top of repeatedly breaking a rule that has a big red background and explicitly warns of a ban, two of jart's invitees were manually warned and banned for other inappropriate self-promotion. Finally, when another of her invitees started obvious sockpuppeting, I looked at the invite tree, saw this pattern of abuse, and cleaned up after it.
To rebut another claim repeated a couple times elsewhere in the comment tree, all banned users get an email notifying them of a ban with the Reply-To header set to my email: https://github.com/lobsters/lobsters/blob/master/app/mailers... None of the users banned in this action have clicked reply to explain how I've made a mistake enforcing an unambiguous rule of ours.
Nobody said they were sockpuppets. Only you are saying that in an attempt to create a strawman and mislead unsuspecting readers. We are not idiots.
The two accounts that were banned for sock-puppeteering were 'i2' and 'thatworkshop'. And you have conveniently omitted these two accounts from your comment. What do you have to say about them? You invited them. And they were sock-puppetting? How do you explain that?
After seeing such wilful blatant misrepresentation, forgive me if I'm having difficulty just taking your word for it.
https://valleywag.gawker.com/why-does-google-employ-a-pro-sl... https://www.dailydot.com/debug/occupy-wall-street-supports-g...
I must say, I respect her a good deal less than I used to.
How can I be legitimately banned for spamming when I made zero comments or posts? I rarely post or comment, so did not get a chance, but I was glad to join the community.
I've done moderating work when phpbb forums had their heydey. The clever sockpuppet accounts or voting rings don't start posting about the projects of their main account right from the first day. The clever ones remain dormant, mostly idle, participating in other discussions. They look just like normal accounts.
But much much later when the main account posts a "show project" post, the sockpuppets or voting rings add +1 or post a few words of praise and then move on to commenting on +1-ing other posts.
And voting rings become more difficult to detect when they don't post anything related, post unrelated stuff but quietly upvote the main account's posts/comments.
Sock puppet is only one possibility. Other possibility is legit accounts forming a voting ring. You and me cant see voting actions. Mods can.
The mods have access to server access logs. Mods can see who +1-ed whose posts/comments. So if you need evidence for sock-puppetry or voting-ringery, try asking the mods about it. We can only speculate here. I'm only describing what I've found in the wild from my days of moderating (small) forums.
I mean to say that only because you can't see the other accounts posting related posts/comments does not make them any more likely or any less likely to be sockpuppet or voting ring. They may be legit. But they could be a ring. Or they may be legit. Only the mods with the access logs can tell.
IIRC over the last year, I've only commented once on lobste.rs, when it came to something about Python's import system, and only upvoted my own blog posts about Cosmopolitan Libc and related comments + maybe one or two other posts. I disagree with the ban, but I can see why my account would get banned as a spammer (one of the accounts banned alongside mine had a double account I think).
lobste.rs is a wonderful community, and I'm happy to see that someone there atleast mentioned this blog post. Hopefully I can get un-banned at some point.
Luckily I can still comment on HN though!
As in, was there extensive testing to prove there are no regressions compared to the glibc version?
I agree with you though -- an extensive testing setup will reveal more things for improvement, be it in my patch or in the libc itself. At the moment I only have direct access to Debian 11/Fedora 35/FreeBSD 13, but I expect we will soon find a nice way to run tests across the different operating systems automatically. One of my ideas was to setup a separate drive or folder with all the executables, accessible from different operating systems/VMs, and then run the tests from each. What do you think?
Here apparently the changes are mostly related to error handling, and I guess that's not something that's very well tested with normal usage.
Also got ripgrep to work: https://github.com/ahgamut/ripgrep/tree/cosmopolitan
Might need to update the build scripts a little to handle the latest updates in Rust and Cosmo (we'll need gcc-11 now), but I expect to get Rust working again soon enough.
Look at the list of constants here: https://github.com/jart/cosmopolitan/blob/master/libc/sysv/c...
Every time I used any of these constants, I'd have to load a whole bunch of them into my binary as a large lookup table, and go through that table every time I needed a check in my program. It might not be that slow, but I believe it would definitely be noticeable.
My goal was to make porting easier without changing a lot of source code in either the libc or in the software I was trying to port, and still produce binaries that are close or better in performance. Under those constraints, this gcc patch seemed like the best way to simplify the process.
If I run into enough codebases where SIGHUP is used as an array index initializer, I will probably attempt your suggestion just to measure the tradeoffs. Or you could try it out and let me know if a separate set of constants is better.
Publishing a list of software that will successfully compile might be helpful. Perhaps this already exists.
The superconfigure script is just a wrapper around the usual configure script used to build your software, supplying flags like --enable-static.
The recent version of cosmopolitan generates ARM binaries for Linux and MacOS (https://github.com/jart/cosmopolitan#arm; mode aarch64). There is also blink that provides the x86-64 emulation layer for (APE and other) binaries on a variety of platforms (https://github.com/jart/blink).
Just created my first ever fat ape executable. I ran `apelink.com -o fat-ape-binary.com o/tiny/examples/hello2.com.dbg o/aarch64-tiny/examples/hello2.com.dbg` to create an executable that runs on Linux+OpenBSD+NetBSD+FreeBSD on x86_64+aarch64.
And x86-64 is a particularly bad bytecode, for one main reason: its strong memory ordering rules, compared to nearly every other architecture. IIRC, simple things like doing a store in x86-64 have an implicit release barrier, while other architectures can freely reorder the stores unless there's an explicit barrier. This means that a x86-64 emulator on for instance ARM has to either be single-threaded (and would still have issues with shared mmap regions), have special hardware support for x86-compatible memory ordering (as found for instance on recent Apple ARM CPUs, and used for their x86-64 emulation, but not common elsewhere), or add explicit barriers everywhere (which kills the performance).
The only real advantage of x86-64 as a bytecode would be that it has less registers than most other architectures (only 16 registers, while other 64-bit architectures usually have around 32 registers), which allows a 1:1 mapping of the registers on the emulator while still leaving plenty of them free for temporary use.
APE is a big “Who needs a VM if an AOT compiled language as low level as C can compile once and run universally?”
Not even a bit of colour, as terminal escape sequences assume a POSIX host environment, and will throw up gibberish if not.
And it was only one example, another one would be any kind of networking, as it isn't part of ISO C.
https://github.com/jart/cosmopolitan/tree/master/libc/sock
..and that also seems to have WinSock support:
https://github.com/jart/cosmopolitan/blob/master/libc/sock/c...
(also Windows is probably the only non-POSIX OS that matters - and APE is obviously only portable to systems it specifically supports, it can't support systems it doesn't even know about, no matter if it is a "POSIX system" or not).
Also cross-compiling is painful; a lot of small OSS projects that don't have a build farm with all of the different BSDs etc. to hand simply don't ship binaries for those. Even e.g. kubectl has a binary for Linux but not for any of the BSDs (unless you count OSX). So this seems like an easy win for those.
It's too big to be build ad-hoc on the user's machine (contains various big 3rd party C++ dependencies, and takes between 2 and 20 minutes to build, depending on the hardware).
Needs to run on macOS x86-64+ARM, Linux x86-64 (ARM would be nice too of course), and Windows x86-64.
I'm currently looking into WASM, but that needs an installed WASI runtime.
I have no idea if there is a tool that actually works like this - I remember an exploit tool that did something similar, but it wasn't quite right - but making that initial bootstrap VM and a useful stdlib an APE executable would also have avoided the "there needs to be a working Python at the other end" issue which persisted in tripping me up for far longer than it should have.
Someone mentioned busybox elsewhere, which sure, will build with this if you disable a whole bunch of stuff, but busybox is supposed to be the minimal utilities needed to run something close to POSIX-compliant, which obviously assumes your system is a POSIX system. It also includes most of the stuff usually provided by util-linux, which obviously won't run except on Linux. If your software is doing something like trying to read from particular device files to get hardware info from the kernel, sometimes that will work on both Linux and BSD (and even Mac since it's kind of a BSD), but it definitely won't work on Windows.
For software that uses the network, how do you query DNS? If you're calling getaddrinfo, I guess Cosmo libc will do some magic for you to make sure that works everywhere, but a lot of software not written in C is doing something not quite that easily made universal, like just assuming /etc/resolv.conf exists and reading it, which definitely won't work on Windows, or using the native platform APIs, which will only work on one platform.
If you distribute desktop software, I would say good luck. Are you going to comply with the XDG? Well, now I guess it will "work" on Windows, but you're not complying with Windows conventions where you're supposed to put stuff in App\RoamingLow or whatever the hell that I would never remember without looking it up every time. Don't try storing data in the registry, because now it will only work on Windows.
Does your software? It better be present on all systems then. Do you query any environment variables? I don't think LC_TIME and LC_COLLATE are provided anywhere but Linux usually. The XDG_ variables are only Linux. Does Windows have PAGER? I don't even know. I'm pretty sure PATH, USER, and HOME work everywhere.
A truly actually portable executable (TAPE) would require far more than an executable file format that can successfully load into memory and execute its first instruction on any platform. There's a reason vendors just write software that uses a browser's JavaScript engine as a platform instead of the OS's native runtime.
EDIT: I don't know if this is ironic, but I guess consider your choice of programming language, too. Go became popular in part because of the static linking, single executable thing, but it inherently can't be cross-platform since it doesn't use the C library and tries to make all calls into the kernel on its own by keeping track of the syscall table provided by every OS it supports. Well, they're still different on every OS, so an executable that works on Linux will not work anywhere else and vice versa.
Out of those 3 only PATH works on Windows.
Windows has a USERNAME environment variable, but depending on your needs you may need to combine it with the USERDOMAIN variable to get the complete username, %USERDOMAIN%\%USERNAME%.
Windows has a HOMEPATH environment variable but it contains only the directory for the home folder. Another variable, HOMEDRIVE, contains the drive the home directory exists on, so the complete home location is %HOMEDRIVE%\%HOMEPATH%. In powershell a read-only HOME variable is set pointing to %HOMEDRIVE%\%HOMEPATH%. The home directory is supposed to store user files. User specific applications, OS components, customization and configuration data should be stored in the USERPROFILE directory. This is very frequently but not always the same as %HOMEDRIVE%\%HOMEPATH%.
Folder redirection is used often in conjunction with remote desktop services, virtual desktops or just to centralize file storage for ease of backup.
> The only way to get reliable message delivery is to email hn@ycombinator.com.
Unless of course you mean that the compiler should be recognizing switches like this, and instead of rewriting them to if trees, it should be rewriting them to switches, changing the labels to use special cosmo specific constants for each of the values, and wraping the input value to the switch with a call to a function that maps the current runtimes platform's values for these over to corresponding cosmo specific constants (and letting other values pass though unchanged).
That... actually might be a simpler compiler transformation to implement. There would be complexity in needing to recognize which of the multiple sets of of constants are being used, and applying the right call or calls to the switch input to map them. It also require complexity on the library side though (creating these ). Not sure if the author would want to implement it, given they have a working implementation of the other transformation.
Lastly such a mapping approach would have risks of inappropriately mapping, or trickiness like having to invert the input before mapping for code that does the negative errno value thing.
Would you consider a code pattern like that to be common? What other codebases apart from busybox have it? If there are many examples, I might spend some time trying to update my patch to handle patterns of that kind.
No way to use magic macros there.
std::error_code is just a wrapper for several different enums. I should have probably just given the posix one, std::errc.
You can't just make the various enumerations of an enum be function calls.
The right implementation would have been to convert error codes from/to posix, rather than just give native error codes and make looking up the reference value a runtime operation.
> Of course, my patch isn’t perfect. It can’t handle some anonymous structs, enums, const ints, or amazing things like using SIGILL as an array index [...]
Is "SIGILL" a typo?
[0] https://support.sas.com/documentation/onlinedoc/ccompiler/do...
I too thought it was a typo when I caused it to occur while building BLIS, but then I found out it was because I had added some AVX thing to the config that my local computer did not have.
I don't understand why the compiler wouldn't just error out if it generates code that will never actually succeed, but I'm sure there's some C programmer's explanation for it.
It could at least warn about it (and if it doesn't have the information to generate a useful warning or error at that point where the ud instruction is inserted, then that's obviously a problem that needs fixing).
https://learn.microsoft.com/en-us/windows/win32/debug/pe-for...
> The name "Portable Executable" refers to the fact that the format is not architecture specific
https://en.m.wikipedia.org/wiki/Portable_Executable
> On Windows NT operating systems, PE currently supports the x86-32, x86-64 (AMD64/Intel 64), IA-64, ARM and ARM64 instruction set architectures (ISAs). Prior to Windows 2000, Windows NT (and thus PE) supported the MIPS, Alpha, and PowerPC ISAs. Because PE is used on Windows CE, it continues to support several variants of the MIPS, ARM (including Thumb), and SuperH ISAs.
In my opinion it qualifies as portable
It also is used by more than just Windows - EFI uses it too. There is no reason in principle why some non-Windows OS couldn’t use it as the native executable format.
And it was a mistake lobbied by Microsoft. UEFI should have used smaller format like TE everywhere. There's no reason for files without imports and other similar needs from the loader to have the complexity of PE. At first it hindered toolchain support at non-Windows operating systems which was probably the real intention.
Working as intended.
> The Terse Executable (TE) image format was created as a mechanism to reduce the overhead of the PE/COFF headers in PE32/PE32+ images, resulting in a corresponding reduction of image sizes for executables running in the PI Architecture environment. Reducing image size provides an opportunity for use of a smaller system flash part.
> TE images, both drivers and applications, are created as PE32 (or PE32+) executables. PE32 is a generic executable image format that is intended to support multiple target systems, processors, and operating systems. As a result, the headers in the image contain information that is not necessarily applicable to all target systems. In an effort to reduce image size, a new executable image header (TE) was created that includes only those fields from the PE/COFF headers required for execution under the PI Architecture. Since this header contains the information required for execution of the image, it can replace the PE/COFF headers from the original image. This specification defines the TE header, the fields in the header, and how they are used in the PI Architecture’s execution environment.
Given it is a modified version of PE, I think it still counts as a member of the PE family of executable formats. It isn't something with a different heritage, such as ELF. Although ELF and PE are ultimately cousins – AT&T invented COFF, and then they invented ELF as a successor to COFF to fix its flaws; other vendors, such as IBM, SGI and DEC, decided to extend COFF to remedy those flaws instead of adopting ELF; and then Microsoft took AT&T COFF and made their own modifications to it to produce PE.
[0] https://uefi.org/sites/default/files/resources/PI_Spec_1_7_A... PDF page 269
Wait, why isn't EINVAL a compile-time constant, shouldn't it just be an enum?
Therefore the resulting code, which must run on any OS, cannot treat it as a constant.
The assumption that constants are constants is a sound assumption, and I highly doubt many would be willing to rewrite the world to make some "clever hack" work.
I specifically wanted to avoid rewriting the world in order to port established codebases like git/curl to Cosmopolitan Libc.
My goal with this gcc patch was to answer the question: "what is the minimum amount of source code I would have to manually change in order to port software to Cosmopolitan Libc?". As we find out, sometimes we don't have to change anything to compile code for an Actually Portable Executable -- just do the usual ./configure && make, and a small `objcopy` command at the end to create the APE.
Right, it's not that it *isn't* a constant, it's that its not the *same* constant everywhere?