liblinux: Architecture-independent access to Linux system calls
github.com
github.com
The part most relevant to this discussion:
> I stopped developing this because I discovered the Linux itself has a better solution that they use for their own tools:
> https://github.com/torvalds/linux/blob/master/tools/include/...
The kernel developers implemented nolibc.h with 3 levels of abstraction: system call entry point, function wrappers and libc interface. They set errno in the libc interface. Personally I don't see the point.
> The third level is the libc call definition.
> It exposes the lower raw sys_<name>() calls in a way that looks like what a libc usually does, takes care of specific input values, and of setting errno upon error.
Defining NOLIBC_IGNORE_ERRNO solves the problem!
> Currently, the only supported architecture and compiler is x86_64 and GCC, respectively
Let's wait until its architecture-independence has arrived in practice.
The Linux kernel nolibc.h already has support and is just better in almost every way. I wasn't sure if there was any point in continuing liblinux after I discovered it since it achieves almost everything I tried to do and doesn't even require a build system, shared libraries or anything. The support for auxiliary values is missing though. Maybe they'd accept a patch...?
I also asked Greg on reddit about nolibc.h once:
https://old.reddit.com/r/linux/comments/fx5e4v/im_greg_kroah...
Turns out there was a klibc project that never turned up in my original research!
* Remember that glibc is part of the GNU project and is meant to be portable beyond Linux, especially to Hurd. It might not make sense for them to implement interfaces that wouldn't work or would require significant work to emulate on Hurd, the BSDs and Solaris.
But since C is the de facto standard for OS libraries, most languages end up supporting calls to C libraries. Thus C libraries do become the language-agnostic library.
So in order to have usable libraries people will wrap their C++ implementation in C interfaces. Rust faces the same problem. All those nice abstractions result in a complex ABI that will never touched by anything outside the Rust ecosystem. There is no escaping the extern "C".
Some languages are virtualized. Java, C#, Ruby, Python, Javascript all have virtual machines. There's no way to interface directly with code running in those languages. We must interface with the virtual machine instead: import it into our process and ask it to call the functions we want to use.
C binary interfaces are not language agnostic but they are quite simple. The result of compiling a C library is symbols pointing to functions with simple calling conventions. As a result, virtually any language can look up these symbols and call them.
The ideal solution is to have a system_call keyword built into the compiler that emits the proper instructions for the target platform.
There have been plenty. What has changed isn't the lack of competing languages, it's the discontent with C and C++. The Rust hype has coincided with the anti-C and anti-C++ hype and thus the "rewrite it in Rust" movement was born. But Rust wasn't even the first memory safe systems programming language, let alone first alternative to C (and neither was C the first systems programming language itself).
You're now conflating "popular" with "better". What makes a language successful isn't always the same as what makes it better (eg there are countless examples of languages failing to gain significant traction (like Zig, Nim, OCaml, D, ) over other languages that had a large company backing it (like Rust did).
Moreover, what makes a language "better" isn't going to be the same for everyone. Some crave simplicity so that the entire language can be memorised (like SPARK) whereas others crave complexity where they can chose the subset of features to support (like C++). Some like C-style braces, some like ALGOL style keywords (like Ada), some prefer S-Expressions (like LISP) and some people even prefer to go direct to assembly.
We've seen operating systems built in Fortran, Pascal, LISP, Oberon and others so it's not even as if I'm making a theoretical point here.
That's definitely true but I think the odds are that at least one of them would have succeeded if many of them really were competitive. They got a lot of chances.
Let's put it in other terms: if your point were true then Javascript wouldn't be the language of browsers. English (which is a mess of a language) wouldn't be the de facto international language.
You cannot conflate "success" with "better". There are so many other factors at play (like money, company politics and which platforms that language is used on happened to luck their way into the mainstream...to name a few).
My point being that the reasons for C's success was as much to do with external factors as it was to do with C itself. In fact in a parallel universe where Bell Labs didn't give UNIX source away and RMS wasn't born (thus wasn't around to create GNU and mandate C as the standard), I'd wager you'd be talking about an ALGOL-derived language (possibly SPARK) as "the best" with C as an academic curiosity; a footnote in history. Or maybe we'd all be writing LISP.
This goes for ISAs too: We use x86 not because it's better or even good. It's because PowerPC (or whatever you prefer) and all those that were better were not better enough to outweigh the massive cost of losing backward compatability, training, habits, relevant capital and all of that. Maybe RISC V, paired with the right microarchitecture, actually does turn out to be so much better that the transition becomes worth that huge cost. We'll see.
We will overcome C, JavaScript, x86 and all that but only once the contenders to displace them are just *that* much better. Many things have had the chance to do that and were better in many ways but not until recently has the balanced tipped, in my opinion.
I'm not making accusations. You literally said:
> That's definitely true but I think the odds are that at least one of them would have succeeded if many of them really were competitive. They got a lot of chances.
Which is the opposite of the point you're now making. I can only respond to the message you post.
Any way, I think we are largely in agreement even if we're describing the problem from different angles.
Both of my points are congruent. Rephrase the quote with "competitive" replaced with "better enough" and this becomes obvious. Your original points were that competitive (i.e. better enough) languages existed and people just got sick of C. I disagreed saying that no, if there had been competitive (i.e. better enough) languages then at least one of them would have succeed by now as they'd all had plenty of chances.
Another argument against this "sick of C" idea is that if that were really true then any of the old languages could have been adopted, now that people are supposedly suddenly sick of C. But that's not what's happening. The new languages are being adopted because they're finally better enough. The old languages aren't being adopted because they were never competitive.
1. There was a lot more around than just BASIC and Pascal,
2. Pascal wasn't born much before C (roughly a couple of years),
3. and BASIC compilers did exist even as early as in the 60s.
Some examples of other languages (all of which pre-date Pascal and C):
Pascal itself was inspired from ALGOL. So there's a whole class of languages you're excluding. ALGOL was typically compiled too.
Also compiled was FORTRAN. In fact the first FORTRAN compiler was written in the 1957 and the language is credited as being the first ever high level programming language.
Even BASIC wasn't one of a kind. There were a few languages similar (the names of which escape me at this point in time). But BASIC was definitely the most popular of it's ilk.
And lets not forget LISP. It's the second oldest high level language after FORTRAN. LISP has had compilers around since the 60s, although the language is older than that.
Another popular language with compilers back in the 60s was COBOL. And as you maybe aware, some banks are still stuck running COBOL code (albeit modern COBOL looks nothing like 60s COBOL)
And I'm just listing the highlights here. I could also talk about how C evolved from B and B evolved from BCPL. But BCPL's syntax is a lot more like ALGOL than it is C. I could also talk about how FORTRAN, ALGOL and COBOL themselves aren't really a single language but rather a class of languages too. In fact Pascal evolved from ALGOL W and ALGOL W compiler was written in PL/360 which is another ALGOL-like systems language designed specifically for the IBM System/360. Which leads nicely onto the next point that there was also a lot of machine specific or vendor specific languages out there too. In fact it was quite common for organisations to design their own language - possibly even more so then than it is now.
So no, it wasn't just a decision between BASIC and Pascal in the early days. There were a lot of choices even back then. And in fact Pascal wasn't even an option in the early days because it didn't come onto the scene until 1970. Verses the 50s and 60s for all the other languages mentioned above.
When microcomputers emerged in the 1970's, in particular consumer microcomputers, many languages and other software simply did not migrate to them from Big Iron, mainly for reasons that it just didn't fit.
People who started computing with microcomputers in that era would only have read about Algol or Cobol in Byte or Creative Computing; it wouldn't be found running on their Comodore Pet or what have you.
That all said, you'd still be surprised at the number of programming languages that were ported to microcomputers, albeit more so in the early to mid-80s rather than the very start of their evolution in the 70s.
Take the British Amstrad CPC 464, that had compilers for Forth, C and COBOL (to name but a few): https://www.cpcwiki.eu/index.php/Category:Programming_softwa...
I remember seeing a similar array of options for other microcomputers too.
But I'm not disagreeing with the GPs point, those solutions I've exampled were pretty niche. I raise it more for academic interest than as a counterargument.
People in the computing field would have known about C in connection with Unix, but Microsoft brought system software written in C to the mass consumer market, smack in the 1980's. That would have raised the profile of C as something proven that could be used to deliver; and you could use the compiler from that same vendor, if you wanted.
I don't know what early versions and manuals you mean because that wasn't in v6 or v7. Was there Fortran on UNIX before F77 was introduced in v7?
So, kernel was written in C and therefore uses C data structures, pointers, and follows C conventions for using them (for example, kernel might return a struct of pointers). This applies to interfaces and structures, making syscalls (depending on which syscall and what is needed), checking return values or error codes.
You don’t NEED this stuff exactly to run on Unix. What and how much you need, or even just convenience, are how it boils down. If you want to use eBPF to trace things from raw network packets, syscalls, file systems, you can do some in Python but you’ll need some degree of C.
You want to run a Django or flask app in just Python? Good to go.
[This article on Linux ABI](https://en.wikipedia.org/wiki/Application_binary_interface) might help explain better/in more detail.
Sorry if this isn’t the best explanation, I’m on mobile and up too late.
But for answering your question deeper, we have to find an understanding what boundary we're talking about. The kernel syscall interface? Things like Linux's vdso? POSIX library (libc)?
The more we go on in that sequence the more C we get into the interfaces and many of those APIs are essential for writing well-behaving non-trivial programs.
The closer we stay to the kernel's syscall interface the less C you get, but still the syscall interface and C are related. Strings are often NULL-terminated. Arguemtns are passed (via registers) from left to right, not like in Pascal convention right to left. (First arg always in EBX, second in EXC etc.)
A different language will need some interoperability with C conventions for being a good citizen on Unix.
A Pascal operating system for instance would use Pascal strings (historically what tis now called ShortString) where length is encoded in the first byte.
Only if the designers would make that decision, and they are not in any way obliged to make such decision! Nobody stops them from using e.g. a moral equivalent of "RECORD size: CARDINAL; str_ptr: ^CHAR END" for passing strings in their system calls (which would probably coincide with using "size_t size, char *str_ptr" in the C prototype at the binary level), or something else entirely.
Is the idea of an OS not exposing its implementation language in its public API that foreign so I have to express it three times in this thread and every time be replied with "No, what are you talking about, of course the OS would use reuse the structures of the runtime of the language that it is written in in its public syscall API"?
Gee, I wonder how the syscall calling convention would look if I wrote an OS entirely in assembly.
With C and Unix the issue of course is that both were developed around the same time by the same people for working together.
"All" languages (which survived) can call into C.
And yes, with common Pascal compilers i can write specific functions in a way to provide a C calling interface (called STDCALL) so those functions can be called from C. But for reaching arbitrary Pascal functions I need my own assembly code which passes parameters in the right order and manages the stack "correctly" (in C the caller clears the stack, in Pascal the calleé)
On x64 Linux, everyone managed to agree to use the same calling convention which is of course very convenient. The language of it's written specification is heavily C-biased, but if you squint you can see that it talks mostly about eightbytes and and compound objects and where to shove those so it more or less trivially translates to most other Algol- or C-descendant imperative languages; I believe it can even be reformulated strictly in language-neutral/assembly terms.
On vanilla x86, of course, everyone invented their own calling convention, even different implementations of the same languages: IIRC, Borland C++, MSVC, Watcom, and GCC on Windows all by default produced object files that you could not simply link together without manual tinkering. And (32-bit) Windows, despite being written in C, uses a variation of "Pascal" calling convention in its system API which they called "stdcall". I believe there were some C compilers that actually used "stdcall" by default instead of "cdecl".
Hell, even on x64, Cygwin GCC (on Windows) uses SysV ABI for everything except interfacing with Windows itself where it obviously has to use Windows x64 calling convention.
[0] http://www.gnu-pascal.de/gpc/Importing-Libraries-from-Other-...
https://github.com/torvalds/linux/blob/master/Documentation/...
On Windows and other operating systems, things such as system call numbers can and do change. There's no way to interface with the kernel without linking to vendor libraries.
So on Linux you can ditch libc and write your own code. A Rust liblinux is definitely possible, I tested it. You can even have a compiler generate the system calls for you! It just needs to support the simple calling convention. No libraries necessary. I've seen a project that does just that:
On Linux, however, we have, for example, glibc on one hand which claims that it doesn't have a goal of forwarding all of the Linux's syscall, and several failed initiatives on the other hand to allow the user's code to perform syscalls only from inside of glibc (which spells doom to anything that doesn't use libc, e.g. Golang) because of security reasons. And the history of still having to support INT 80h together with SYSCALL on x64 on the third hand.
I didn't know about these efforts. Please elaborate.
The problem, again, is not that this kernel-entry interface is factored out into a separate shared library; the problem is that it's glued into a runtime of one specific programming language that now apparently everybody has to use.
Yeah. I don't like that. Just because that's the way it works now doesn't mean it must work that way forever. Because of its stable system calls, Linux doesn't impose any design on user space. People could theoretically rebuild the entire system in Lisp or Rust or something. Potential is limitless. Also, it's amazing that I can compile a freestanding program with zero dependencies and boot Linux straight into it.
> the problem is that it's glued into a runtime of one specific programming language that now apparently everybody has to use
I agree. It's a really hard problem to solve. Since it's based on the POSIX APIs there are tons of definitions and data types that are specified exclusively in C and its preprocessor. Not exactly accessible for other languages. The preprocessor macros in particular are extremely problematic. The only sane way to untangle those is to use a C compiler.
There's an interesting discussion in that thread you linked. What if the vDSO was the entry point for all system calls? That would certainly be interesting.
I think the vDSO ABI is underspecified. It's just the C ABI, whatever that means on your system. This can cause bugs due to assumptions. For example, this Go bug happened because Go called hardened versions of the vDSO functions with insufficient stack space:
https://news.ycombinator.com/item?id=28411786
Compared to the C ABIs, the Linux system call ABI is really simple and easy to understand. No way to screw it up.
The vDSO ABI should be as simple as the system call ABI. As it is right now, it pretty much forces people to use C.
The basic idea is to make it harder for remote code execution bugs to result in useful abilities. Restricting access to syscalls to libc, and using adress randomization to make it hard to find libc makes it harder to do system calls when you take over a program.
https://lwn.net/Articles/806863/
> For static binaries, the valid regions are the base program's text segment and the signal trampoline page.
I'm not sure but it looks like the BSD libc is some kind a special case? Surely this is not necessary? A compiler or language runtime should be able to somehow mark those segments with the appropriate permissions.
https://github.com/sunfishcode/mustang
It supports threads, filesystem, networking, and more!
The library for making Linux system calls is:
You could always call syscalls directly, using syscall() (https://man7.org/linux/man-pages/man2/syscall.2.html), but a wrapper library is nicer.
One of my objectives was to produce ELFs with no symbols other than those defined in my own code. Glibc adds a lot of symbols.
Ohh, if only we could make C the second class citizen. C brings with itself meaningless legacy from 1969 into modern systems, like you might think openat2 was created recently, but it's designed for PDP-7, because it's C and stuff was always done this way.
What replacement are you proposing that every language can call into? How would a migration of existing Java, Python, Ruby, C++ etc programs be done to this new interface? Do programs need to be modified at the source level? Just recompiled? How will the existing programs that cannot be recompiled be handled?
I quite aware that you can make the C ABI a shim into the new first class ABI to make existing programs work, but why would anyone target the new ABI when the existing one still works?
Okay, then it's aimed at people who want to use more than just POSIX, right? That's the entire world.
In which case the question still applies - how do you migrate everyone over to your new interoperable ABI? How will I run Gimp, Firefox/Thunderbird, IntelliJ, VSCode, Slack, Xterm, all my command-line tools (tar, grep, etc), Git, Meld, Starcraft 2 (and all 100+ games in my Steam library, and Steam itself) and more?
How do you see your replacement running MySQL, Apache, Nginx, PostgreSQL, bash/zsh/etc, SQLite, stunnel, netcat, wireshark/tcpdump, etc?
Do they all have to be recompiled? Replaced with new sootware?
Because each and every one of the software I listed above depends on a C ABI. And any shim provided to make existing software work will simply result in developers ignoring your new ABI and targeting the C ABI for maximum compatibility.
Every Linux process interfaces through the kernel via the system call interface. It is stable and language agnostic. The interface is defined at the processor architecture level.
https://github.com/torvalds/linux/blob/master/Documentation/...
https://man7.org/linux/man-pages/man2/syscall.2.html
Any compiler can emit this sort of code, including JIT compilers! It's possible to discard the entire C user space and rewrite everything in Rust. Would be a monumental effort but still.
Note that no matter what there will be at least one C library remaining: the vDSO the kernel maps into the address space of every process.
https://github.com/torvalds/linux/blob/master/Documentation/...
https://news.ycombinator.com/item?id=25997506
Apparently for BSD, the ABI is not that stable, glibc works around bugs on some platforms (adding memory barriers etc), and there are security features that check the origin of syscalls. I wonder whether this is truly acceptable on Linux...
This is not a problem on Linux.
> glibc works around bugs on some platforms (adding memory barriers etc)
That's really interesting. Do you have examples of such fixes? I'm more familiar with musl sources, glibc proved to be somewhat difficult to navigate.
Well, yeah, but that's a tautology, no?
A willingness to discard everything indicates a willingness to rewrite everything. So, yeah, it is possible to rewrite everything if you are willing to rewrite everything.
Linux lets you do this. The interface is stable. People can build their custom user space on top of it and enjoy a great kernel full of drivers. Wouldn't it be amazing if someone came up with a 100% Lisp user space? A 100% Rust user space?
Great indeed (I like Lisp!), but I have to question how useful that would be in practice.
Let's take Lisp as an example: I can now program only in Lisp and talk to the OS in native Lisp only (including POSIX calls). The downsides are that I cannot call any existing library; my 100% Lisp environment cannot use libzip, or libffmpeg/libavutil, or libSDL, etc.
In order to make that 100% Lisp environment actually useful it's going to be doing FFI calls anyway into a C ABI. That makes the OS-interface redundant.
Python is as popular as it is purely because it made calling into the C ABI easy: existing libraries were made instantly useful in Python, not unnecessarily hard.
"access to a very limited subset of Linux system calls" :P Still a cool project though and a good base.
make system-calls.missing