Obviously, do whatever you want if you’re just playing around, but please do be aware that the syscall interface on macOS is private, subject to change at any time, and should never be used directly.
for all things bad about windows, one cannot say that they didn't take backward compatibility quite seriously
https://j00ru.vexillium.org/syscalls/nt/64/ (click show all)
This is of course a little annoying at times.
When people talk about the supposed requirement to depend on the platform libc dso, what they're actually talking about is Apple frowning upon statically linked binaries. https://developer.apple.com/library/archive/qa/qa1118/_index... They don't forbid it though, like Microsoft and Fuschia do. Fuschia for instance uses RIP origin detection. Windows does it by changing the RAX ordinals fortnightly. Apple simply asks that we say, hey, if you use Cosmo there's some risk Apple might break our binaries. We take proactive steps to avoid that happening with Cosmo. For example, we don't do some of the things Go did, like reverse engineering the memory layout of Apple's time functions. Cosmo sticks to the APIs that are shared by UNIXes in general, e.g. gettimeofday(), rather than depending on Apple's own internal designs, e.g. Mach system calls. I don't believe Apple can rightfully claim APIs that aren't their own, as being their own implementation detail which they can change at will. We do our best to respect Apple's boundaries, so I believe the risk of breakage with Cosmo on Apple should be minimal.
Apple doesn't differentiate between "UNIX" and "Apple" when it comes to API; a particular API is either public with stability guarantees, or it is private and unstable.
> I don't believe Apple can rightfully claim APIs that aren't their own, as being their own implementation detail which they can change at will.
Apple absolutely does claim this, and if that means the library ABI has to change to accommodate a change, the dynamic linker and symbol tricks are leveraged to keep things working for code built against the earlier ABI.
If an engineer comes up with a really clever trick to make gettimeofday() just a tiny bit faster, but this requires breaking syscall ABI, they will absolutely do that.
> I believe the risk of breakage with Cosmo on Apple should be minimal.
Using system-private interfaces on Apple platforms means there are no guarantees here. You'll be OK, sometimes, for some releases. It mostly worked for Go, for a while.
It's not "two megacorps" disagreeing, it's "supported interface" vs "unsupported, unstable, system-private interface".
On paper. Have they acted on it? Have they actually broken an API that used to come standard with UNIX?
The underlying syscalls that support them have occasionally changed and broken existing apps that bypassed libSystem. For example, Sierra broke all go apps that called `gettimeofday` (the exact syscall jart used above an example!) because the go compiler emitted direct syscalls: https://github.com/golang/go/issues/16606
I'd like to know, did Apple actually break a "private" C ABI when this ABI actually implemented a standard UNIX function? I know they could, but did they?
By standard UNIX function I take it that you mean a function defined via POSIX and part of one of the various specifications used for UNIX certification (for the moment lets ignore the fact there are multiple revisions and optional extensions). It is important to note that the specifications says essentially nothing about:
* Binary formats
* Libraries (static or dynamic)[1]
* What symbols are in what library
It is all written in terms of what source code should compile, and how that compiled code functions. Everything else such as calling conventions, syscall interfaces, what is library code vs a syscall, etc is an implementation detail.
So given the above, I am not entirely sure what you mean by a `"private" C ABI when this ABI actually implemented a standard UNIX function.` Do you mean has Apple ever changed an internal function called by a function specified in POSIX? IOW, if your question is does Apple reserve the right to implement `stat()` as a call to `stat_internal()` and then change the arguments to "stat_internal()" ? Absolutely.
If you mean has Apple ever changed a function that is part of POSIX but it considers private? Those don't really exist on macOS, if POSIX allows it and it is part of the standard that has passed conformance it is by definition public and the C ABI level interfaces for as exposed by libSystem are stable (which is not to say that all of those interfaces are great, but they are standard and supported). IOW, the standard specifies that `stat()` exists, and it is by definition public.
That is not to say incompatible changes have never had to happen (for example, when UNIX conformance was originally implemented a lot of existing functions required incompatible changes to pass the test suites). All of that is handled via symbol versioning and redirecting new binaries to different symbols than the older binaries used, which maintains both binary compatibility for old binaries and allows new source to compile in the correct (conformant) way. This is why if you inspect libsystem_kernel.dylib you see variants of symbols like:
* _recvmsg
* _recvmsg$NOCANCEL$UNIX2003
* _recvmsg$UNIX2003
The old ones keep working with the existing semantics for older binaries, the headers have magic in them redirect to the newer ones when targeting the appropriate minimum OS version, and the userspace libraries have multiple entry points that provide both sets of semantics (often implemented in the userspace shim, sometimes by dispatching to the kernel with different syscalls).
[1]: Despite that, at this point POSIX does specify some of the semantic of `dlopen()` and `dlsym()`, which is pretty insane when you think about.
If you by "allowed" means "is meant to work", then yeah it means it's not allowed on many of these platforms.
OpenBSD has infrastructure to control which memory regions syscalls are allowed to stem from, specifically designed to block syscalls from anything by libc. See https://lwn.net/Articles/806776/
I do not recall if it was enabled, but when someone does not provide you an ABI it very much means that you are not supposed to try to write code against it. If you do it anyway, you will have to live with the resulting instability.
There are no alternative libc implementations for *BSD, unlike Linux the system provided libc is the only libc that matters.
If OpenBSD wanted to, they could implement something like io_uring with support for all kernel functionality, port libc to use that and ditch conventional syscalls entirely (simulating blocking where needed), without user-space knowing anything changed.
Under Linux, which is actually rather unusual in providing ABI guarantees, you're stuck maintaining bug-for-bug compatibility for every syscall you've ever written, and cannot change user-space no matter how good that change would be for either or both.
But instead you have to maintain bug-for-bug compability in libc for every API you've ever written. In case of macOS where the kernel and libc/libSystem developments are closed and done by the same entity is makes zero difference (imho).
Also, I understand catiopatio's argument in the sibling comment about doing more in user-space than in kernel in case of a bug, but it breaks down the moment thin wrappers around syscalls exist there. You link to libc/libSystem and use every thin wrapper in existence - now no syscall can be changed (if I understand correctly how macOS works, never used one)
It is not instead. With syscall ABI you need to maintain both, without it you only need to maintain libc (a majority of which is dictated by POSIX anyway).
> In case of macOS where the kernel and libc/libSystem developments are closed and done by the same entity is makes zero difference (imho).
As above, maintaining two contracts is harder than one.
Proprietary parts aside, most OS's have their kernel and user-space developed together, and that is exactly what allows them to work this way.
This is used to adopt new features, change or deprecate old features with a brutal efficiency that Linux cannot compete with - e.g., when OpenBSD implemented pledge in both kernel and all relevant tools.
This is one of the reasons that these projects can keep up or in some cases surpass Linux (FreeBSD networking is seen as superior, and you used to get better performance from running Linux binaries on FreeBSD through its compatibility layer) despite having much smaller groups of maintainers and users.
There's not really a strong argument for not linking libc. Even if you want to implement your own libc, you can still do so with linker tricks and calling through to the supported syscall wrappers in the real libc.
Sure there is. The libc sucks. Freestanding C actually turned out to be a superior language because there's not as much legacy weighing it down. There's many systems languages out there, nobody should be forced to link to C stuff.
While you cannot link directly to libsystem_kernel.dylib, if you choose to ignore everything in libsystem_c.dylib and only use the syscall wrappers reexported from libsystem_kernel.dylib via libSystem it has practically the same effect on macOS (in fact, the resulting binary will be identical to what a hypothetical binary linked to just a single libsystem_kernel.dylib would be except for a single `LC_LOAD_DYLIB` command).
Such a binary would have the system lib c initialized sitting in its address space, but for the rare binary that really wants its own libc that doesn't seem unreasonable.
It's annoying that libc combines two completely different things - userspace utility functions, system calls - but unfortunately, that's where history has brought us.
Not on Linux.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
The system call interface is not only stable but also language agnostic. The correct place for this inferface isn't in some C library, it's in the language itself. We could have compilers directly targeting this. GCC could add a system_call keyword that emits code conforming to that ABI. Dynamic languages could have a JIT compiler that does the same thing.
That said, the vDSO is much smaller than any libc.
However, if, for some reason, you really want to re-implement libc, you still can, but your library will have to link against the system-provided libc for the syscall entry points, because that's the only stable interface to the kernel.
There's no reason that should be a problem for a hypothetical libc-reimplementor.
In this case that means the api designer gets a little bit of wiggle room in the process userspace where it might be more appropriate to say shim some calls so that they're no longer 1:1 with a syscall. The obvious example is that malloc() can call syscalls for you and is the "default" way to allocate memory you might want to provide but It's more of it's own little runtime.
My favorite example was in my time at LinkedIn. Because the internal Kafka team had thick client libraries, making the company wide change to enable encryption was as simple as pushing out new libraries and deprecating old ones.