Actually Portable Executable
justine.lol
justine.lol
(Unless you’re Intel/AMD in which case please fight it by giving us faster, more power-efficient chips for cheaper. Thanks!)
And the past, I mean PowerPC was a thing for a long time on both Apple desktop systems and servers.
Glad that the compiler toolchains make this transition a lot easier, and Apple has been a major contributor to that. It helped ease the transition from 32 bit ARM to 64 bit ARM, it enabled easy cross-platform apps (Mac Catalyst) and now from x86 to ARM for desktop apps.
And of course it's not particularly new; 25 years ago (ish) Java came out that promised the same thing, one codebase that runs on all architectures. Scripting languages, too.
Apple doesn't have to worry about 35 years of legacy architecture to support.
X64 decoders are indeed only ~5% of the die on a modern CPU, but it's 5% that is always at 100% utilization. That's a non-trivial amount of extra power. X64 decode parallelism is also limited. I've heard four instructions at once as a magic number beyond which it becomes really hard. This is why hyperthreading (SMT) is so common on X64 chips. It's a "cheat" to keep the pipeline full by decoding two different streams in parallel (allowing 8X parallelism). SMT isn't free though. It drags in a lot of complexity at the register file, pipeline, and scheduler levels, and is a bit of a security minefield due to spectre-style attacks. All that complexity adds more overhead and therefore more power consumption as well as taking up die space that could be used for more cores, wider cores, more cache, etc.
ARM is just a lot easier to optimize and crank up performance than X86. The M1 apparently has 8X wide instruction decode, and with fixed length instructions it would be trivial to take it to 16X or 32X if there was benefit to that. I could definitely imagine something like a 16X wide ARM64 core at 3nm capable of achieving up to 16X instruction level parallelism as well as supporting really wide vector operations at really high throughput. Put like 16 of those on a die and we're really far beyond X64 performance in every category.
This is also why SMT/hyperthreading doesn't really exist in the ARM world. There's less to be gained from it. Better to have a simpler core and more of them.
IMHO X86/X64 has hit a performance wall at least in terms of power/performance, and this time it might be insurmountable due to variable length instructions and associated overhead. It matters in the data center as well as for mobile and laptops. There's a reason AWS is pricing to steer people toward Graviton: it costs less to run. Power is the largest component of most data center costs.
L1 size in 2021: 64 kB
For the L1 instruction cache, aliasing shouldn't be a problem (since it's never written to), but this is once again another place where the x86 legacy hinders things: instead of requiring an explicit instruction to invalidate a virtual address in the instruction cache, it's implicitly invalidated when writing to that address.
256KB L1I/128KB L1D
Little cores: 128KB L1I/64KB L1D
When I use M1, AWS Graviton, or even older Cavium ThunderX chips I can't help but think that X86 is on its way out. The advantage is something you can subjectively see and feel. It's obvious, especially when it comes to power consumption.
Process node has something to do with it, but it's not the whole story. I'm typing on a 10nm Ice Lake MacBook Air and while this chip is better than older 14nm Intel laptops it's still just shockingly crushed by the M1 on every metric. 10nm -> 5nm is not enough to explain that, especially since apparently Intel is more conservative with its numbering and Intel 10nm is more comparable to TSMC 7nm. So it's more like TSMC 7nm vs TSMC 5nm, which is not a large enough gap to account for what seems to be at least 1.5X better performance and 3X better power efficiency.
Some of the X86/X64 apologists remind me of old school aerospace companies dissing not only SpaceX and Blue Origin but the whole idea of reusable rockets, trying to convince us that there's little economic advantage in reusing a $100M rocket stage that consumes ~$100-200K in fuel per launch.
"That's not much of a meteorite. It's no big deal." - Dinosaurs
The biggest problem is all the REX prefixes. The inefficient encoding of registers in x86-64 squandered all the advantages that x86 had.
> > x86 generally has an advantage [empahsis added, not "x86-64"]
Obviously if you take the worst of both worlds (bloated and variable-width instructions), you can squander that advantage, but the advantage is in fact real.
The good news is, once those boundaries are found, uops can be generated. But that ~5% or so of die space is always running full tilt (provided there’s no pipeline stalls).
I’m sure Intel and AMD have put a massive amount of work into theirs to make it as quick as possible,[b] but it’s still ultimately a sequential operation.
With RISC-like architectures like ARM and RISC-V, you don’t need that boundary detector. Just feed the 2 or 4 bytes straight into the decoders.
[a]: Unlike ARM and RISC-V which have fixed 2 or 4 byte encodings (depending on processor mode), x86’s instructions can be anywhere from 1 through 15 bytes.
[b]: Take the EVEX prefix for example. It is always 4 bytes long with the first one being 0x62. So, once you see that 0x62 byte after the optional “legacy prefixes”, you can skip 3 bytes and go to the opcode. But then you need to decode that opcode to see if it has a ModR/M byte, decode that (partially) to see if there’s an SIB byte, decode that to see if there’s a displacement (of 1, 2, or 4 bytes), etc. And then, don’t forget about the immediate (which can be 1, 2, 4, or (in one case of MOV) 8 bytes).
As for why it became an issue, instruction sets need to be designed from the beginning to be forward expandable. Intel has historically not done that with x86. Take AVX for example. Originally, it was just 128 bit (XMM) vectors encoded as an opcode with various prefix bytes being used in ways they weren’t intended. Later, 256 bit vectors were needed. So they made the VEX prefix. But it only had 1 bit for vector length. This allowed 128 bit (XMM) and 256 bit (YMM) vectors, but nothing else. So when AVX-512 came along, Intel had to ditch it and create the EVEX prefix and allow both to be used. But EVEX only has 2 bits for vector length. So, should something past AVX-512 come out (AVX-768 or AVX-1024?), it’ll probably use the reserved bit pattern 11, and they’ll be stuck again if they want to go past that.
For an example of this being done right, ForwardCom[0] (started by the great Agner Fog) took the “forward compatibility” (hence the name) issue into mind and used 2 bits to signal the instruction length. It’ll probably never reach silicon, but it and RISC-V (which is in silicon form) are good examples of attempting to keep things forward compatible.
The compiler would know the instruction boundaries. It could store that information in a read-only section in the executable. The OS would then just pass that section to the CPU somehow.
I don't think there is anything impossible about this. Would there be sufficient performance benefit to justify the added complexity? I don't know, quite possibly not.
Ultimately these discussions are quite hard because AMD aren't on exactly the same density, and Intel are quite a way behind at the moment.
Apple's use case is "run applications". It's not like there's any magic or they have some sort of ultra specific workload they improved by 10x while the rest sat there.
Apple's customisations are largely "throw hardware at the problem", which I'm reasonably sure Intel would do if that worked for x86. So sounds like something you can do with ARM, which you can't with x86.
The more magical customisations are workload specific, but then they would only trigger for these workloads, both of which are pretty much opt-in: running emulated x64 code on ARM, and performing matrix computations (which AFAIK will only be used through the Accelerate framework).
There's not reason why Intel couldn't, but they don't have the incentive to hyper-optimize frequently used Apple workloads like Final Cut Pro.
If Intel could they would, for years now they’ve been spending billions to get fraction of a pc improvements on benchmarks. You really think if they could increase die size by 10% and get 30% better perfs they’d say no? Come on.
> they don't have the incentive to hyper-optimize frequently used Apple workloads like Final Cut Pro.
Except M1’s performance improvements show up across the board including software which has no relation to Apple, so this is just complete nonsense.
Portability is not hard. If you write standard C/C++ that does not depend on undefined behavior (like wild-ass pointer casts, etc.) you will be fine 99% of the time. Use newer languages like Go and Rust or higher-level languages and you won't even notice.
The only hard areas where labor intensive porting is needed are hand rolled ASM or the use of CPU-specific extensions like vector code (e.g. __m128i and friends). That's a tiny fraction of code written and is generally confined to things like codecs, graphics engines, crypto, and math kernels.
The author precisely realises that there used to be multiple architectures, just like you, but also notices that we have converges on x86-64 - what she terms the lingua franca.
I also completely follows her sentiment, that we should not switch ISA unless there is a very real computation per power unit benefit of doing so.
This is not my area of expertise, so I'm not the one to write the rebuttal, but it seems that "---- is not hard" is never anything more than an invitation to someone who fully understands ---- to explain why it is hard.
Succinctly, serving 99% of the use cases with no effort mainly seems to be a recipe for making sure that, when one hits those 1% problems, one has no idea how to deal with them. I suspect that portability is one of those things where it's easy to do a mediocre job but hard to do a good/robust job.
Apple isn't scattering anything by running processors that run the same architecture as android and ios on phones. Most open source software can already be compiled for x86, arm, sparc, etc.
The stuck-in-the-'90s "desktop is all there is" mindset is a weird holdover from the early growth of PCs in developed countries. If you look at emerging markets, mobile is completely dominant.
ARM has also been historically used most often on proprietary systems you need authorization to develop for. So it's made less sense as a target for open source tooling hack projects like this one.
ARM also has so many sub-targets that it's almost like a coalition of ISAs rather than a unified one like x86. So adding ARM support to Actually Portable Executable might not be as simple as including an ARM build in the binary. We might need to have multiple ARM builds for its microarchitectures. Because ARM users want resource efficiency and they're not going to be happy with a generalized build that broadly targets ARM; they want code that's narrowly targeted to the specific revisions of the processor that they're using.
In other words, we can't give ARM users portable binaries because ARM users do not want them.
I also always thought that code for other architectures was the kind of thing that mostly got contributed by the people who build those architectures. Things like how IBM always graces our GitHub issues with patches each time our code doesn't work on s390x mainframes. I like that they do it by contributing patches rather than the feedback of why don't you support this? Why don't you support that? Oh I didn't say I actually needed it.
(Rule 5: if you haven't played SC2, Banelings were a super in-character addition to the zerg unit tree. Zerglings are able to morph into a variant called the "baneling" that's a fairly powerful suicide bomb. Rather like the "scourge" air units in SC1, except a ground-only attack. Fairly good at rushing bunkers, but die pretty fast to siege tanks.
Really loved the design though - fit the zerg ethos really well.)
Nitpick: it is by definition not needed to have different build for different microarchitectures (except maybe for performance). If the same code doesn't work on different chips, it's because they have different ('versions of') instruction set architectures.
Edit: nevermind, on rereading, you were already complaining about that. You should probably stick scare-quotes on "'need'", though.
Apple already does this for x86: macOS contains duplicates of all operating system binaries & libraries compiled for pre- vs. post-Haswell processors.
Back then IBM Mainframes still had as strong foothold in large corporate IT departments. Sun had a dominant position as well for most newer companies. If you wanted multiple CPU's with redundant fail over and gigs of RAM Sun was your huckleberry back in the day.
It’s trivial nowadays to write a program in Go or Rust and deploy it to whatever architecture you want, without any arcane knowledge of the build process
According to rust docs [1] and go wikipedia page [2] both have mainly support for x86, while go recently added support for macos/arm and in 2019 windows/arm, rust only has tier 1 ("guaranteed to work") support for arm-linux and x86.
Am I misreading this? It does not seem "trivial" to me for arbitrary platforms.
[1] https://doc.rust-lang.org/nightly/rustc/platform-support.htm...
[2] https://en.wikipedia.org/wiki/Go_(programming_language)#Vers...
I do my job at work every day on a Tier 2 ARM target, and in practice, don't notice any difference from the Tier 1 targets. YMMV of course.
Could've written a win32 program.
Show HN: Redbean – Single-file distributable web server - https://news.ycombinator.com/item?id=26271117 - Feb 2021 (141 comments)
... which is still high on the front page. Also there was a big thread last year, which is still within the dupe window:
αcτµαlly pδrταblε εxεcµταblε - https://news.ycombinator.com/item?id=24256883 - Aug 2020 (286 comments)
We downweight follow-ups because otherwise the front page gets too repetitive and repetition is mainly what we try to avoid here:
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...
:)
Just a smiley, no other words!
I'm invoking them with 'bash -c'.
sudo sh -c "echo ':APE:M::MZqFpD::/bin/sh:' >/proc/sys/fs/binfmt_misc/register"
I don't think it's that big of a deal either, since compiling these days is fast enough you can do it 3 times without it being a problem.
Don't get me wrong, it's very impressive, I just don't think it makes that big of a difference in practice, especially since environmental differences will still require you to have 2 codebases in many scenarios (like accessing the filesystem for example)
The change here would basically be that all versions of it would have to be compiled into the same binary, with a runtime switch.
I think the importance of ISAs will fade away generally in favor of specifications that enable coordination of higher level memory model semantics "across" compute resources -- the cpu/compute core becomes the thing that allows you to share reads and transfer write ownership of memory as efficiently as possible between heterogeneous components that operate on the compute graph -- and many of these compute components may require various binary forms of task specific instruction encoding ...
There is a tcsetattr function in Cosmopolitan.
If I use that to obtain character-a-a-time input with no echo, and then run the portable executable Windows, will that have the right effect in the console window?
In under twenty or so fairly simple commits to a fork of the Cygwin DLL, I got it:
http://www.kylheku.com/cygnal/
This is another useful tool in this general arsenal.
Just to give some context, I am not a Python person, as I prefer Go. But given the popularity and the suited use-cases I think it is a good option.
Second issue is that Python without any extra modules is still pretty limited, nearly every serious python project comes with some extra dependencies.
So have a Python interpreter available only gets you so far...
(For the record, the same could be said about pretty much all dynamic languages I'm familiar with).
Contrast:
cat test.txt | grep search
import os
import subprocess
with open('test.txt', 'r') as f:
for line in f:
line = line.rstrip()
subprocess.call(['/bin/grep', line, 'search'])
I will give you this: bash variables and expansions can be confusing. Contrast with programming where this probably works:
"start" + variable + "end"
By default return codes of failed commands are silently ignored and `set -e` does not work under all circumstances. By default every variable is part of the global scope and the best you can do about it in a POSIX compliant way are sub-shells, which in turn have no way to change variables outside of their scope.
It is just broken by design :-/
The opposite being true for your average general-purpose program, where managing tasks is a secondary concern and delegated to a library.
The bash version looks for the pattern `search` in every line of `test.txt`, the Python version treats `test.txt` as a file of patterns, and look for each of these patterns in the file `search`.
And of course you wouldn't implement the bash version in python that way as it's rather trivial to do it in Python:
out = [line for line in open('test.txt') if 'search' in line]
As a Greek, if you do this, I hate you. Why the hell do you have to make me read "actmally pdrtable execmtable"? At least this is one of the less offensive cases.
EDIT: Solidarity to our Cyrillic friends!
> it's good for readers to have to work a little
Unless they're using assistive technologies. In that case it's a nightmare. Don't make your users work.
> it's not hard for any HN reader to do the bit of work to figure it out
Unless they're using assistive technologies. Or just want to read it without work.
Or, say searching for it. This post comes up. The one you linked to doesn't.
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
Respect to you for wanting to honor Greece. I think using the letters* correctly would honor them more. (thanks for the correction)
(Greece doesn't use Cyrillic but I agree with you otherwise)
So I'd say that the SVR4 source code would be a good place for you to start. It's like ambrosia and once you've read it you can always tell by reading modern code which developers have and haven't seen it. There's also the Lions' Commentary on Unix. I highly recommend Richard W. Stevens. The last book on the required reading list is BOFH.
I've been studying this for a while. Turns out Linux has an amazing interface. It's stable and language-agnostic. All you need to do is put the values in specific registers and execute a special instruction. The result comes back in one of those same registers.
The high level documentation is here:
https://man7.org/linux/man-pages/man2/syscall.2.html
https://man7.org/linux/man-pages/man2/syscalls.2.html
https://www.kernel.org/doc/Documentation/ABI/stable/syscalls
https://www.kernel.org/doc/Documentation/ABI/stable/vdso
On Windows there is a similar interface but it is not stable. The system call numbers can change. Developers are supposed to use the good old Microsoft DLLs in order to get anything done. Just like how everyone uses libc on other systems.
Linux is different. The system call binary interface is the Linux interface. So it's actually possible to trash all of GNU and rewrite the entire Linux user space in Rust or Lisp or whatever. It doesn't have to be written in C. It doesn't even have to be POSIX compliant. Could be GUI-focused!
All you need to make any x86_64 Linux system call is this code:
long
system_call(long number, long _1, long _2, long _3, long _4, long _5, long _6)
{
register long rax __asm__("rax") = number;
register long rdi __asm__("rdi") = _1;
register long rsi __asm__("rsi") = _2;
register long rdx __asm__("rdx") = _3;
register long r10 __asm__("r10") = _4;
register long r8 __asm__("r8") = _5;
register long r9 __asm__("r9") = _6;
/* r8, r9 and r10 may be clobbered but can't be in the clobbers list
because the compiler won't use clobbered registers as inputs.
So they're placed in the outputs list instead. */
__asm__ volatile
("syscall"
: "+r" (rax),
"+r" (r8), "+r" (r9), "+r" (r10)
: "r" (rdi), "r" (rsi), "r" (rdx)
: "rcx", "r11", "cc", "memory");
return rax;
}
Once I realized this I tried to turn it into a library called liblinux... I stopped working on it when I found out the kernel already has an awesome single file header you can include that lets you build freestanding Linux executables for a ton of architectures. They use it on the kernel to build their own tools!
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
It even includes process entry point code! Linux copies the argument and environment vectors to the stack before entering the executable. The process start up code obtains those pointers and passes them to the main function. It also ensures the exit system call is called.
The process entry point is usually called _start because that's what linkers look for by default. In reality the ELF header has a pointer to the program's entry point, the actual symbol doesn't matter. You can tell the linker to set it to any other address or symbol. Also note that it's an entry point, not a function. There is no return address. Allowing that code to terminate results in a segmentation violation. Hence the need to ensure exit is called before that happens.
The only feature that seems to be missing is support for the table of auxiliary values:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
The auxiliary values are placed on the stack immediately after the environment vector. So all you need to do to find this pointer is loop through it until it goes out of bounds. I wrote this code and it works:
https://github.com/matheusmoreira/liblinux/blob/master/start...
struct auxiliary { Elf64_Off type; Elf64_Off value; };
static void *after(void *vector)
{
void **pointer = (void **) vector;
while (*pointer++ != 0);
return pointer;
}
int liblinux_start(void *stack_pointer)
{
long count;
char **arguments;
char **environment;
struct auxiliary *values;
count = *((long *) stack_pointer);
arguments = ((char **) stack_pointer) + 1;
environment = arguments + count + 1;
values = after(environment);
return start(count, arguments, environment, values);
}
https://github.com/matheusmoreira/liblinux/blob/master/examp...
> You would like this project: https://chromium.googlesource.com/linux-syscall-support/
Yes, I would! I saw references to this library in your source code, specifically your jump slots implementation. I had no idea Chromium had this and I've been meaning to explore it later. I'm gonna do it now.
> Because too many people who post here hold the viewpoint that SYSCALL is evil and you must link the platform libc dynamic shared object or else you're a very horrible person who deserves to have their binaries broken like Apple did to Go.
I know what you mean! Using system calls are heavily discouraged by libc maintainers and even users. Using calls like clone will actually screw up the global state maintained by glibc threads implementation. It gets to the point where they don't even offer wrappers for system calls they don't want to support. I don't like it... What's the point of an amazing system call that lets you choose exactly which resources you want to share with a child task if all it's ever used for is some POSIX threads implementation?
Even the Linux manuals do this for some reason: the documentation I linked in my above post actually describe the glibc stuff as if it was part of the kernel and leaves the actual binary interfaces as an afterthought. Linux manuals also inexplicably host documentation for systemd instead of a generic description of how a Linux init system is supposed to interface with the kernel. It makes no sense to me!
I even asked Greg Kroah-Hartman about it on Reddit:
https://old.reddit.com/r/linux/comments/fx5e4v/im_greg_kroah...
I actually think using the system call interface is better than using the C library. No thread local errno business, no global state anywhere, no buffering unless you do it explicitly, no C standard to keep in mind... It's just so simple it's amazing. It's also stable unlike other operating systems which ship user space libraries as the actual interface. On Linux there's no reason not to use it!
> I feel like you should publish this and post it on Hacker News.
I wrote a liblinux library, the README describes part of my journey learning about this system call stuff. Lots of LWN sources!
https://github.com/matheusmoreira/liblinux/blob/master/READM...
I've been thinking about expanding on it in order to describe everything I know about the Linux system call interface. You really think I should publish this?
The reason I didn't post liblinux here is it's in a very incomplete state and actually less practical than the kernel nolibc.h file. I only discovered the header much later into development and figured there was no point anymore since the kernel had a much better solution not only available but in actual use. I ended up rewriting autoconf in pure makefiles instead...
For example if you want to call fork() using asm() then on Linux it's simple:
int fork(void) {
int ax;
asm volatile("syscall" : "=a"(ax) : "0"(57) : "rcx", "r11", "memory", "cc");
if (ax > -4096) errno = -ax, ax = -1;
return ax;
}
int fork(void) {
char cf;
int ax, dx;
ax = IsLinux() ? 57 : 2;
if (IsXnu()) ax |= 0x2000000;
asm volatile("clc\n\t"
"syscall"
: "+a"(ax), "=d"(dx), "=@ccc"(cf)
: "1"(0)
: "rcx", "r11", "memory", "cc");
if (cf) ax = -ax;
if (ax > -4096) errno = -ax, ax = -1;
if (ax != -1) ax &= dx - 1;
return ax;
}
I used to value portability but now I believe in using Linux everywhere and for everything. I like OpenBSD too but Linux is the stable one you can build anything on. What I wanted to eventually accomplish is a 100% freestanding Linux user space with no libraries at all. Maybe boot straight into the program I want to use, just like we can pass init=/usr/bin/bash in the kernel command line. How far could this go? Using nothing but system calls it's actually possible to get a framebuffer and use software renderering to draw some graphics. I'm guessing pretty far.
By starting from scratch like this it's possible to fix all the historical problems with our systems. For example, I think it's unacceptable when libraries keep global state. This can't be fixed without getting rid of libc and its buffers and caches and errno. Removing this cruft would actually simplify a threads implementation. And then there's completely insane stuff like .init and .fini sections:
https://blogs.oracle.com/solaris/init-and-fini-processing-wh...
A similar statically-linked user space project I found years ago:
If you want to honor Greece, use the letters as they're meant to be used! "Acτuaλλy πoρτabλe εxecuταbλe" would be much better (though I've intentionally tried to give English readers a stroke with this one :)!
How does this do anything to honor the cultural legacy of Greece? Perhaps we could honor the legacy of 19th century mathematics by using Fraktur characters when they resemble Latin ones?
When people who can read Greek are telling you it's bad taste maybe take their word for it! Not dang.
What you're really saying is that the Greek alphabet (and by extension its language community) is so insignificant compared to Latin that the cost of potential misrecognition is so low that it can be disregarded. This is chauvinism, not "honoring Greek mathematics"!
I agree with dang's feelings/thoughts about the issue.
Perhaps a solution would be to add the "normal" meaning between parenthesis, after the one in greek alphabet?
What are you going to work on in the near future? Curious to hear about it. If you don't want to post in public, $my_hn_username at gmail
αcτυαlly pοrταblε εxεcυταblε
EDIT: For completeness, the full transliteration (or as close to it) would be "άξουαλι πόρταμπολ εξεκιούταμπολ". The extra "o" in "portabol" and "execiutabol" is actually a schwa, I think, so it can be omitted.
Edited: I missed the pi and rho completely though, my bad
Actmallu pdrtable execmtable
If it is only stylized as "αcτµαlly pδrταblε εxεcµταblε" a readable name should be available alongside the visual styling (using ARIA attributes, for example).
The `title` attribute should specifically not be used here; ARIA labels should. The title attribute is implemented differently across browsers and assistive technologies [1] and is supposed to be a title, not a duplicate of the content of the element.
https://developer.mozilla.org/en-US/docs/Web/HTML/Global_att...
Яцssiди Сдяs
And it’s just so horrible to make a title using Cyrillic characters according to what looks like Latin and not according to their actual sounds XD
There's a subreddit for parodying the phenomenon - http://reddit.com/r/grssk