Hacker News Reader

Putting aside the personal jab, which the other comments have rightfully called you out for (really, this is why people detest “the orange site”) you’ve just fundamentally misunderstood the point of the blog post.

C is kind of a horrible way of specifying machine interfaces, because it tries to lift things into being portable when the interface is anything but. The ABI is what ties C to the specific machine interface that it’s exposing, and in doing so dooms the ABI from ever changing. The article has many, many examples: a simple one is that it’s stupid to expose a 32-bit value as an “int” because that means “int” needs to be 32-bit forever. In this sense providing a C API is bad because the interpretation of the API into ABI is dependent and on your tooling and sometimes even that doesn’t agree, with fun™ results. This very much isn’t a request for “please give Rust a nice interface to the kernel” but more a “there ought to be something better than having to run libclang just to talk to the kernel”.

So what do you suggest instead? Express APIs in yet another language (perhaps one with defined-length numeric types) other than the ones people actually use? An IDL? Who's going to do that work, from the definitions themselves to the bazillion language-specific bindings? I'm not saying that you should have to have an answer before you can criticize, but every solution to this particular problem sucks in some way. Some things are just hard. Jumping on a bandwagon to tear down something that already exists, ignoring or trivializing any problems it does solve or that alternatives would need to, is another much-despised Orange Site behavior. Nobody ever turns out to have a real silver bullet in their back pocket.

Fuchsia/zircon uses IDL to define the system API interface and it works for them :D

Ok to be fair Fuschia could define their system API using Protobuf and it would work for them

:p

This was more an observation that you can specify a system ABI in a way that isn't a fragile as C's. At least it isn't xml plists :D

Well, you can tell why the orange site put it on the homepage, right ;) But to respond in seriousness: yes, the blog post exists to mostly highlight a problem, and it doesn’t have an easy solution. But there’s a couple things you can do to help: one is to define a sort of “restricted C” that is easy to parse and designed to be ABI stable, which I think Microsoft actually tries to do. You could use some sort of IDL, yes: web browsers do this for their “system interface”, and as ‘olliej mentions so does Fuchia. But the main point is to shed some light on the issue to start a discussion, because until now people have mostly been OK with defining everything using C and it’s been causing problems that nobody’s really thought about trying to solve, and I think the post does a pretty decent job of that.

It's less that the OS is written in C, but more that literally every single interface to literally anything is either written in C, or isn't meant to be used by multiple languages. C is the de facto standard for interacting with anything not written in your language. Window management, I/O, graphics, everything has a C interface. There aren't even any alternatives (besides just sending a list of bytes through a pipe or something, which noone wants to do for obvious reasons).

I don't disagree, I see an expectation on the part of driver/API providers that you can write (probably in C) and adapter layer that can talk "glorious language" on the input, and then do the naughty bits and talk to the C API on the output.

My position is that this is less of a burden than writing something that goes from the "glorious language" into machine code.

Understanding why that is less of a burden is gained by writing a compiler where you go right from "expressing what you want" to "machine code that does that".

I mean, a C interface is better than a machine code interface. The problem with that is that, unless you include a C compiler (or at least a thing that understands the 176 C ABIs) with your language, you can't use it. (not that machine code would be better at that, but I'm sure that if people actually intentionally attempted to make a global cross-platform ABI, they'd easily make something much better than C)

> a C interface is better than a machine code interface

The last thing I want when I write code in assembly is to have to call a C library just to invoke a system function.

You'd obviously prefer calling into things made in your own language than some other. But, from any other language you'd much prefer a C interface over having to explicitly describe which registers & stack slots to put things in and read from.

I did this once trying to mitigate Go's FFI performance cost. It wasn't great but it wasn't awful either. If I had to do it again I'd generate the assembly trampolines.

> Go's FFI performance cost

https://news.ycombinator.com/item?id=17165179

There are a couple solid summaries in that thread based on my experience going through that exercise - thanks for the link

In case anybody wants to go down the rabbit hole of shit you should definitely not use in production (but we did anyway), this was my starting point: https://words.filippo.io/rustgo/

I did not, however, `no_std` or avoid heap allocation on the Rust side. Everything worked great, including running a multithreaded tokio runtime.

Still do not recommend in prod ;)

> My position is that this is less of a burden than writing something that goes from the "glorious language" into machine code.

I disagree. Machine code has a small stable surface area. Compiling to a bootable image for a single-tasking machine is easier than compiling to a well-behaved userspace unix program that operates the way users expect.

> My position is that this is less of a burden than writing something that goes from the "glorious language" into machine code.

It's only less of a burden if you're guaranteed to have a C compiler hanging around any time the C ABI breaks. If you aren't guaranteed to have a C compiler hanging around any time the C ABI breaks, then you are probably just as well off generating C-ABI compatible machine code from an ABI description written in "glorious language" and updating it when the C ABI breaks.

Non self-hosting languages distributed as source code can just lean on the C compiler for everything.

It's also quite feasible to target Linux without worrying about the C ABI, as the Linux system call interface is famously stable, but that is not true of any BSD.

> but more that literally every single interface to literally anything is either written in C, or isn't meant to be used by multiple languages. C is the de facto standard for interacting with anything not written in your language.

Even that is not the real complaint, since the alternative to this (N different FFI-style APIs) is madness.

The heart of the complaint is "great, we have effectively a single interface to almost everything, which is probably a good idea, except that the interface sucks".

> C is the de facto standard for interacting with anything not written in your language.

Not necessarily. C is how your language interacts with the OS if you use the OS's C library. But not all languages do. Go doesn't.

The only thing you have to do to interact with the OS is make system calls. You can do that without going through the C library. It might be a PITA to do it if you're not working on Go for Google, but if that's the real problem, that's what the author of this article should have complained about. (It's actually less of a PITA on Linux because the syscall interface is implemented using software interrupts. Windows is more of a PITA because the OS goes out of its way to make its syscall interface look like a C interface.)

Again, there's more than the OS kernel that a programming language will need to interact with. You don't go through syscalls to make an X11 window or read keyboard/mouse inputs or draw graphics. There are thousands of utilities that are meant to be usable cross-language, but in actuality use the C ABI (well, at least one of the 176 C ABIs there are).

> C is how your language interacts with the OS if you use the OS's C library. But not all languages do. Go doesn't.

...on Linux. Pretty much everywhere else, libc is the way to interact with the kernel; even Go goes through libc on ex. Darwin and OpenBSD, because Linux presenting a stable kernel ABI is actually pretty unusual.

> Linux presenting a stable kernel ABI is actually pretty unusual.

Did you mean "portable"? Because the ABI to the Linux kernel system calls is very stable. Literally the 32-bit syscall interface has not changed (except by addition) in 30 years.

The darwin kernel is also pretty stable, except Apple moves so fast from one architecture to the next that they drop support for old ISAs pretty fast. I predict they'll drop support for x86 altogether in ~5 years.

> Did you mean "portable"? Because the ABI to the Linux kernel system calls is very stable.

I think the GP meant that the Linux syscall ABI is indeed very stable and always has been, as you say, but that the syscall ABI of other OSs is not. So Linux is "unusual" in that sense, not in the sense that its syscall ABI is stable now but wasn't in the past.

Sorry, yes, I misunderstood the comment.

AFAICT, SunOS (Solaris), BSD, and Darwin have stable kernel ABIs for the core (UNIX) system calls. I've never used the Mach syscalls on Darwin, but I can imagine those changing more often, because Apple does that. I would imagine that Windows supports 32-bit syscalls still pretty well...but I stopped using Windows in 2002, so tbh, not sure.

read is unlikely to change anytime soon on macOS but Apple can and does change its numbering for lesser-used but still very important system calls all the time.

They also change the syscall abi without touching the numbering. gettimeofday(2) is a pretty famous one there.

Then there’s also the Linux-specific issue of vDSO, which are not kernel code and to which the article applies in full (see: “debugging an evil ho runtime bug”).

I have a very different understanding of the situation; the best description of the situation I can find is https://utcc.utoronto.ca/~cks/space/blog/programming/Go116Op... (discussed at https://news.ycombinator.com/item?id=25997506 ) - from that article,

> The official API for Illumos and Solaris system calls requires you to use their C library, and OpenBSD wants you to do this as well for security reasons (for OpenBSD system call origin verification). Go has used the C library on Solaris and Illumos for a long time, but through Go 1.15 it made direct system calls on OpenBSD and so current released versions of OpenBSD had a special exemption from their system call origin verification because of it.

Darwin has always explicitly not had a stable kernel ABI (https://developer.apple.com/library/archive/qa/qa1118/_index...). Go tried to bypass the system libraries on macOS and it resulted in things breaking, so now they do use system libraries (https://github.com/golang/go/issues/17490).

As sibling comment notes, yes Linux has a stable ABI, and amongst all operating systems that is incredibly rare.

>.. libc is the way to interact with the kernel..

On UNIX clones, written in C.

Win32 isn't libc, nor the mainframes ones are (language environments), or Android (Java)/ChromeOS(Brower APIs).

> Win32 isn't libc,

I'm less familiar with the Microsoft ecosystem, but I'm pretty sure it is; Microsoft named their libc MSVCRT.DLL, but it's still the canonical system ABI, and NT, AIUI, actively shuffles system calls between builds to prevent people even trying to use them directly.

The rest, and your general point, are fair, though; there's nothing preventing the creation of a system that doesn't work like this.

No, msvcrt.dll isn't the system ABI. It's officially not even part of the OS and you aren't supposed to rely on it existing. Kernel32.dll is the primary entry point for OS things, and it's distinctly not a libc.

MSVCRT is short for "Microsoft Visual C (++) Run Time".

It is not an API to the OS.

Even Go gave up and switched to using libc for OpenBSD[1]. You can use syscalls on Linux because Linux, lacking a blessed libc implementation, guarantees stability of syscalls as an interface. The BSDs, afaik, do not, so circumventing libc is liable to render your binary forwards-incompatible with future kernel versions.

[1] https://go.dev/doc/go1.16#openbsd

> The BSDs, afaik, do not,

That isn't just about the lack of guaranteed stability, as far as I understand it is considered a security feature. System calls that are not made through the libc may be actively blocked. So an attacker can't just use an exploit to inject a system call into a process, the call has to pass through the libc wrapper and in combination with ASLR that wont be trivial.

A better way of accomplishing this would have been to randomize the syscall numbers on a per-process basis, and map a "syscall number translation table" into the process when loading the executable.

> The only thing you have to do to interact with the OS is make system calls. You can do that without going through the C library.

If by "the OS" you mean "Linux". On Windows, calling syscalls is explicitly unsupported and Microsoft will break you mercilessly by changing syscall IDs between versions. On macOS and iOS, libSystem.dylib is the only supported way to call syscalls; syscalls are SPI and Apple will break you if you try to call them directly. BSDs likewise discourage you from calling syscalls.

The "syscalls are stable API" principle that Linux is famous for is a weird Linux-specific thing; other OS's don't do it.

Not since Windows Server 2022 and Windows 11.

To enable the same kind of container image portability between kernel versions that Linux has, Microsoft has decided to stabilise the kernel syscall ABI.

I'd like to read more about this. Do you have additional resources that you can link to?

>> C is the de facto standard for interacting with anything not written in your language.

> Not necessarily. C is how your language interacts with the OS if you use the OS's C library. But not all languages do. Go doesn't.

Go doesn't interact with anything written in a different language? I find this really hard to believe - it would result in Go not having a GUI library, not being able to use PostgreSQL, not being able to start external processes (like shells), etc.

I'm pretty sure that Go can do all those things, hence Go does interact with libraries written in a different language.

> Go doesn't interact with anything written in a different language?

That's not what I said. I said Go doesn't interact with the OS using the OS's C library. But, as others have pointed out, that's only true on Linux.

You should think of C as a front-end for the computing platform you are targeting. The platform has an ABI, which you need to use to interact with the system components. So because every third-party has to speak that ABI to interface with the platform, they might as well use that ABI to interface with each other too. You can of course build your own little world within a platform, with all your binaries speaking a different ABI to each other. And that might have some advantages, but most people find those advantages outweighed by just using the same ABI as the platform.

> Window management, I/O, graphics, everything has a C interface.

You say that like it's a bad thing. The situation has gotten much worse for third-party languages these days, with frameworks being written Javascript, Swift, etc., which are hard to bind to without fundamental impacts on your language design.

Yes, it is possible to make worse ABIs than C's. That's not a surprise. Doesn't mean C's is automagically the best possible. Something that, at the very least, doesn't have varying sizes for the same type between architectures/OSes, and precisely defines structure layout/padding, would be vastly better. (not saying that's a realistic thing to get at this point, but doesn't change the fact that it'd be much nicer.)

I’m not saying it’s the best thing, but I’m not sure you can get much better. A lot of things defined by the C ABI are things you’d have to do anyway if you’re compiling to native code (structure alignment, where’s the stack, what’s called-save, callee-save, where do you find the return value, etc.) Targeting the C ABI’s set of choices isn’t that different from targeting a different set of choices.

And there are gratuitous differences in C ABIs, for sure. But often there’s good reasons. X86 had no register arguments because few registers; same reason it had no thread pointer. RISC architectures had weird limitations on unaligned access, etc. affecting struct layout.

Maybe I’m being dense. What does a good alternative look like? CORBA?

Between architectures you'd indeed need quite a few differences, but for the same architecture it'd be nice if things more or less agreed on things.

Specifically, I think it'd be very reasonable & possible to have an ABI that has various floats & integers (specified width of course), and pointers & structures, that has consistent conservative padding on all architectures and precisely one way any given structure is laid out in memory (ok, maybe two, with LSB & MSB integers; but I think that's literally all the variation there is (and MSB is nearly dead too)).

Not the most efficient thing on all architectures, but for language inter-communication it shouldn't be too bad to lose a couple nanoseconds here and there, and I'd guess it'd outweigh the alternative of needing more generic (and thus, less optimized) code handling each individually.

the big issues with C are that it doesn't specify the size of it's integer types.

The various C ABIs definitely do, though. And you generally know which C ABI you need to interact with (the __int128_t example from the article just seems like a bug -- luckily use of that type is rare).

If I could design my own cross-language ABI, I'd make these differences from C:

* Use fixed-size types instead of the 5 types for 4 integer sizes (is long 32-bit or 64-bit?)

* Allow multiple return values [so you could use multiple return registers, especially on non-register-starved arches]

* Allow unwind ABI (C has no way to permit unwinding!)

* Dedicated types for pointer + size for arrays

* Dedicated string type (i.e., distinguish between &[u8] and &str in Rust terms)

* Something that allows for vtables would be nice

* Something that encodes ownership (and eventual deallocation) is also potentially valuable

Where is the C interface to Symbolics Lisp machines…?

There’s lots of alternatives, MSR wrote an OS in C#

A lisp machine, by definition, runs lisp. The ABI problem only exists in environments where multiple programming languages exist.

I doubt much non-.NET software runs on that C# OS either.

If you want multiple programming languages to be able to sanely run within a single OS (and do more than just pure computation), though, you have the ABI problem, and this is what the whole discussion is about.

The Symbolics lisp machines had C and Fortran compilers

Implemented in Lisp.

.NET was created as a Common Language Runtime.

When it was announced it supported about 27 languages.

From Microsoft themselves, J#, C#, VB.NET, and Managed C++ (later replaced by C++/CLI).

It was WebAssembly before its time (among many others), with tons of features that WebAssembly is yet to support.

Genera definitely had a C (and I think C++) interface.

Written on top of Lisp.

I think that's besides his point, which I take is: Replace C with an alternative and people will complain of similar problems, since all abstractions are leaky abstractions to some extent. Or in other words, since you can never be immune to this "leaky" problem, you will inevitably have similar kinds of problems despite using an alternative. Which is why you'll find every programming language has critics.

Also, keep in mind having alternatives isn't necessarily better, as it introduces a problem itself: it complicates things (the article even mentions a problem of this sort). Does having 100 programming languages, with dozens of OS's, each all doing mostly the same sort of things in different ways - simple? Might the programming world be simpler if there was 1 programming language/OS/way of doing something, rather than having 100s of alternatives? (This is just food for thought to demonstrate the point, not something I'm particularly advocating)

> the "I like computers but don't really understand computation" fallacy

> I always encourage people who are "learning computers"

Buddy, the author is a major contributor to the Rust project

does that preclude one from making bad points in a rant?

I think it precludes one from “not really understanding computation”.

no but even though this is written in a rant-ish style it isn't isn't really one given that the post is full of specific examples discussing all the things that are wrong with C as an interface.

This is probably the most technical post I've seen on HN in a while and accusing someone who goes fairly deep into the implementation details of compilers, and OS libs of 'not understanding computation' is somewhat bizarre.

> Phantomderp and I have both recently been very aligned on a particular subject: being extremely angry about C ABIs and trying to fix them.

Damn beginners!

I think it's been a few years since Gankra has been active, but she was the original mastermind of std::collections as well as the original author of the Rustonomicon, if anyone wants her credentials.

She’s still on the website under lang-docs, at least: https://www.rust-lang.org/governance/teams/lang#lang-docs%20...

As well as a former engineer on the Swift team at Apple. Aria is more qualified to talk about compilers than 99% of HN.

> I always encourage people who are "learning computers" to actually write a compiler

Does the Rust compiler count? The author has been a long time contributor.

> I kind of think of this as the "I like computers but don't really understand computation" fallacy.

You couldn't be more wrong about the author

I only made it about 80% through because, yes it was a rant. I'm pretty sure the author knew what they were talking about. The issues discussed were those of someone who knows how computers work with a great deal more depth than many (if not the vast majority) of developers. The point that programming languages are just syntactic sugar around machine code isn't really relevant here since, for better or for worse, C headers are the standard way to codify access to ABI's. A deep understanding may be fine if you aren't accessing many ABI's, but it doesn't scale well. Using C headers does scale well, but the author is pointing out there are many things about C that makes this difficult.

The most difficult part of C-headers-as-interfaces is the use of macros as part of the ABI.

Agreed. That part is just mind-bogglingly awful, and fairly unique to C/C++. AFAICT all the other problems are common across most alternatives, many of which are actually worse.

> "I like computers but don't really understand computation"

You know you could make your point without being condescending and trying to insult the author.

The author invites insult by being insulting themselsves, not to mention comically overweening.

They are free to write a new os in rust with superior abis if they want to show us all how it's done.

I'm sure it will be in use by the entire world in 50 years the way c is today, and no one from that time will write a blog post like this one because he of course will get it right.

No software is going to be perfect. But new software can at least try and improve upon the old software.

Do you really think that Unix and C are the absolute pinnacle that our field has to offer, we shouldn't bother trying anything else?

(Not a unix or C hater, just a bit of a tangent about how it's cool that people make new things even if they won't be perfect).

>> Do you really think that Unix and C are the absolute pinnacle that our field has to offer, we shouldn't bother trying anything else?

Unix and C are not the pinnacle of ease of use, but rather the survivors of the past 50 years of programming language and operating system evolution. So many of their parents, relatives, competitors, and even children have died and yet they remain.

That means that they are the most adaptive and the most successful in a Darwinian sense, but not necessarily the most ergonomic or user-friendly.

That would be more compelling if their success was more directly coupled with their technical design.

Unix is not technically incompetent (certainly given the needs of the times it was designed in), but its real success now is largely based on it already being an incombent, and being open enough for projects to work from.

Original Unix is dead, it lives on through copy-cats, and they copied it mostly because an experienced user-base makes adoption easier.

In other words, the only time Unix was primarly selected for technical reasons was very short lived. Now its technical benefits are little more relevant than the technical benefits of a qwerty keyboard.

>> That would be more compelling if their success was more directly coupled with their technical design.

>> Unix is not technically incompetent (certainly given the needs of the times it was designed in), but its real success now is largely based on it already being an incombent, and being open enough for projects to work from.

I disagree.

The technical design of Unix and C as the implementation language were major factors to its success and versatility.

Eric Raymond describes the design choices and culture of Unix that helped make it successful:

* Rule of Modularity: Write simple parts connected by clean interfaces.

* Rule of Clarity: Clarity is better than cleverness.

* Rule of Composition: Design programs to be connected to other programs.

* Rule of Separation: Separate policy from mechanism; separate interfaces from engines.

* Rule of Simplicity: Design for simplicity; add complexity only where you must.

* Rule of Parsimony: Write a big program only when it is clear by demonstration that nothing else will do.

* Rule of Transparency: Design for visibility to make inspection and debugging easier.

* Rule of Robustness: Robustness is the child of transparency and simplicity.

* Rule of Representation: Fold knowledge into data so program logic can be stupid and robust.

* Rule of Least Surprise: In interface design, always do the least surprising thing.

* Rule of Silence: When a program has nothing surprising to say, it should say nothing.

* Rule of Repair: When you must fail, fail noisily and as soon as possible.

* Rule of Economy: Programmer time is expensive; conserve it in preference to machine time.

* Rule of Generation: Avoid hand-hacking; write programs to write programs when you can.

* Rule of Optimization: Prototype before polishing. Get it working before you optimize it.

* Rule of Diversity: Distrust all claims for “one true way”.

* Rule of Extensibility: Design for the future, because it will be here sooner than you think.

Source: http://www.catb.org/~esr/writings/taoup/html/

It was not openness alone that contributed to the success of Unix. There were other open systems, but they did not survive.

What's the part where you disagree?

Yes, those things did contribute. Originally.

But once Unix won, all it needed to stick around was momentum.

There’s a difference between making (valid) complaints about the state of things and presuming that you (or anyone!) could fix them single-handedly

It not being possible to improve the situation doesn't mean that the current situation is good. Is the article's rant gonna achieve much? Most likely, nope. I'm fairly sure the author recognizes that.

> I always encourage people who are "learning computers" to actually write a compiler (there are some good starting points for that but online courseware from MIT and other sources can get you the lecture material too). Doing that helps broaden one's perspective of what language designers and implementers are up against with regards to pretty much every computer working the "same" way (Von Neumann or Harvard architecture wise)

I suggest looking up the author's CV.

Total foot in mouth, here... hope you are having a good day, otherwise!

>I always encourage people who are "learning computers" to actually write a compiler (there are some good starting points for that but online courseware from MIT and other sources can get you the lecture material too). Doing that helps broaden one's perspective of what language designers and implementers are up against with regards to pretty much every computer working the "same" way (Von Neumann or Harvard architecture wise)

Isn't it dependent on what IR do you use?