How Swift achieved dynamic linking where Rust couldn't (2019)
gankra.github.io
gankra.github.io
It's rare to read a corporate story where a company traps itself with technical choices so thoroughly, and still manages to pull through in the end. Much respect for the Uber engineers.
So you rarely save disk and every app has different shared objects loaded, and even assuming you use one compiler for everything, it may not matter at all. If anything it's worse because it can negatively impact startup time. In practice, ~everyone compiles Haskell code statically, and only dynamically links to libraries that have exposed a stable ABI. I'm not surprised people do the same for Rust.
Also, shared common package library units for releases by default shrinks binaries rather than having zillions of exhaustive, slightly-different combinations of similar structures and machine code hoarding space in binaries. I've never understood how 300 MiB binaries would ever be acceptable to anyone, especially going against decades of shared libraries that reduced binary sizes and memory usage. Deliberate waste is never acceptable because there is no free lunch.
I wish Rust had been around 30 years ago when I loved doing low level programming. Between my C++ books and gigs, C++ was good financially but Rust is just better (apologies to people who love C++).
I think Rust’s killer feature is lifetime analysis by the compiler. In 1991, you had 33 MHz 80486 with around 4 MB of RAM. I don’t think the average developer had the computing power to do Rust’s lifetime in any reasonable amount of time.
Rust lifetime analysis is a function-local pass. It never needs to access any state outside the function. It can be done on all functions in parallel, etc.
You could do Rust alias analysis with 33 MHZ.
Secondly, a lot of the benefits of dynamic linking don’t make as much sense now.
First with regards to space savings. With modern disk capacity having extra copies of executable code is probably not that big of a deal. In addition with Link Time Optimization, you actually may not be saving space with dynamic linking since the linker can pretty aggressively throw away code that it knows is ever called.
Second, with regards to security. One thing to consider is that dynamic linking is so complex that it is one of the reasons why people use docker, to make sure their binary and its dependencies are stable. Once you have something in a docker, it has a lot of the same update issues as a static linked binary, only less efficient.
In addition, with security, Rust unlike C and C++ is a memory safe language. Given the vast majority of security issues are memory safety issues, I would expect Rust to have a lot fewer CVE’s. I think the experience with Go is likely enlightening in this regard. I am not aware of a huge issue of lack of security because it didn’t do dynamic linking and you couldn’t do security updates as easily.
Finally, Rust as a new language is embracing the new way of programming and deploying. We are shifting away from using binary artifacts (often closed source) that once built were rarely changed, to build from source and continuously build/deploy. Cargo makes it relatively easy to do this. In that model, updating a binary for security is just a subset of the normal building and updating of a binary that is done daily.
By avoiding dynamic linking, I think, Rust positions itself best as the no-compromise, high performance, modern, systems programming language.
Without a dynamic linking story, Rust is just not viable for writing UIKit, Android's frameworks, etc. Which is OK, it's a reasonable choice, but it limits Rust's scope.
With static linking, you get to prune at the function level at link time. DLLs can't do that.
It doesn't matter if it is the kernel or your application that calls dlopen().
How do imagine that something like ld.so gets implemented?
Everyone else has to go through dlopen, LoadLibrary, or whatever alternative the OS offers.
Which is exactly the same code path that at some level the OS dynamic linker relies on.
If you are statically linking everything, then there are no libraries to give input to dlopen anyway, it doesn't load .a files, and .so is not what we want to support anyway.
But lets keep the assumption that in the world of static linking advocacy, explicitly loading dynamic libraries is acceptable.
SO now, not only has the application to do the job of mapping dynamic library symbols to function pointers, they are constrained to C ABI function calls, which is not Rust anyway.
As for you Linux based assumption, there are OSes, where dynamic linking doesn't take place before the PC jumps to main, you can do deferred dynamic linking. The call sites are marked for triggering a page fault on call, which only when it happens, will the OS look around for code to load.
This is how managed languages like Java and C# work, which many keep forgetting there are also AOT compilers, just in case the VM arguments comes into play.
But since we are talking about systems languages, you can find this feature on Windows and Aix, just to give two examples how the OS world isn't everywhere a Linux clone.
https://docs.microsoft.com/en-us/cpp/build/reference/linker-...
https://www.ibm.com/support/knowledgecenter/ssw_aix_71/gener...
My application calls mmap and mprotect(PROT_EXEC), not dlopen.
I'm not familiar with the inner workings of dlopen, but I wouldn't be surprised if dlopen itself is mostly userspace code that knows how to interpret the ELF format, and internally it uses mmap as the actual system call. Maybe it has its own system call, though, to let the kernel do most of the work?
I've seen some really cool uses of mmap. It's nice for reducing the number of memory copies when you're churning through a bunch of data on disk, and often necessary when using shared memory for low overhead inter-process communication. I worked on a project that needed publish-subscribe message passing semantics with fine-grained synchronization across dozens of processes, for example, and a lock-free shared memory "disrupter" queue structure was able to handle millions of messages per second, while sockets could only handle a few hundred thousand or so.
I've seen some unfortunate uses as well. Sometimes folk like the idea of manipulating files on disk as arrays in memory, and use mmap even though there's no practical performance need for it. The code often times ends up harder to read than it would have had it used file descriptors. Also, the performance ends up worse because the implemented algorithm's access patterns unknowingly cause lots of page faults and disk seeks because the data is an "array". Often times there's an alternative way to do the same work sequentially with an additional in-memory data structure, and using file descriptors naturally biases one towards doing that.
There are plenty of OSes to choose from by the way, even your rant barely scratches the surface of the options available for any serious systems programming language.
If a OS (or alleged OS) demands that you dynamically link against libc (or anything else written in C) to interact with it, that problem is not the (non-C) programming language's fault.
> your rant barely scratches the surface of the [OS] options available
That's fair, but most of the not-linux I have experience with is microcontrollers or other things that are far enough from unix/posix that they don't have dlopen (mmap itself is hit-or-miss, but tends to be hit in the sorts of situations where I'd have occasion to dynamically load stuff in the first place), so there's not much I could do about that.
And even then good luck getting everything to work without issues.
Ah the wonders of having a language server microservice running across the network, or a image conversion plugin or audio DAW.
I think this is an insightful comment. Obviously there are other reasons to use docker, but too often (cough...Python...cough) I see virtualization used for exactly that.
Coding for almost 40 years and I am yet to use Docker to sort out this kind of issues.
> I think the experience with Go is likely enlightening in this regard. I am not aware of a huge issue of lack of security because it didn’t do dynamic linking and you couldn’t do security updates as easily.
When I started programming, dynamic linking was only available on big iron machines filling computer rooms, we did not need Go for knowing what static linking entails.
> Finally, Rust as a new language is embracing the new way of programming and deploying. We are shifting away from using binary artifacts (often closed source) that once built were rarely changed, to build from source and continuously build/deploy. Cargo makes it relatively easy to do this. In that model, updating a binary for security is just a subset of the normal building and updating of a binary that is done daily.
If Rust wants to succeed in replacing C and C++ in typical big corp, cargo better support binary libraries eventually.
I am quite sure WebSphere, just to give an example, isn't releasing the source code for the .so that come along its Java implementation.
You can use static linking in c++ if you want to. There is 0 advantage in not having the option to use dynamic linking if you want to. It might be even more secure if e.g. the program links dynamically against a commonly used library and this library is updated regularly.
The advantage is in other people (eg libc maintainers) not having the option to require you to use dynamic linking when you don't want to.
While you can do static linking in C++ if you want to, you still pay the cost for support for dynamic linking because of ABI issues. You can static link, but you still lost out on new language features and changes that break ABI and so you are still stuck with sub-optimal standard library code. For example, for years, until GCC 5, GCC std lib had a non-standard, slower copy on write implementation of std::string, that stuck around for so long because it would break ABI to change it. Even now, in MSVC STL, there are a lot of performance optimizations which the maintainer acknowledges, but will have to wait because they would break ABI. So even if you just statically link and continuously build in C++, you are still paying the price for support of dynamic linking.
Visual Studio 2017 - 2019 were the first time they ever bothered with it, plus it is planned that VS vNext will break it again.
This not counting the other C++ compilers on Windows with their own standard libraries, like C++ Builder and Intel C++.
There is always COM/WinRT if I want an OOP ABI that doesn't break as easily.
But if you are dealing with OS level stuff, talking directly to hardware, in the embedded space, etc, now you care about different ABIs for performance reason. Now you decide, do I want static linking or dynamic linking? The only place I see dynamic linking really being a feature is when you have to link against the kernel, drivers, or proprietary subsystems (e.g. CUDA). IMHO, most userspace application layer stuff shouldn't be sharing memory, it should be communicating, again using byte streams and serde.
So that basically leaves the hardware/software interface: kernel APIs, drivers, etc. If you're in this space, great, it makes sense to use dynamic linking with very stable ABIs with no generic shenanigans, you don't need them. I think a lot of pain of DLL hell comes from using dynamic libs when you really should be statically compiling if it's a library, and IPC if you need to communicate.
The GCC compiler for Java (GCJ) started out doing whole-program compilation, but that couldn't handle Java's flexible ABI and dynamic loading. In 2004 they introduced a compilation mode with an additional layer of indirection similar to Swift's: <ftp://gcc.gnu.org/pub/gcc/summit/2004/GCJ%20New%20ABI.pdf>. They didn't have to deal with monomorphizing, though.
Go's does something similar to Swift to avoid code bloat in their built-in polymorphic map: <https://dave.cheney.net/2018/05/29/how-the-go-runtime-implem...>. It's just a one-off manual thing, though.
- A library can add new fields to a struct.
- A library's parameterized types can be instantiated with new arguments.
Skimming over the D ABI docs, the compatibility seems much more rigid.
edit nvm I thought dynamic linking slowed things down since static linking means code fits better in the cache. Looking at the points, it seems that either dyanmic linking makes code faster or people prefer slower dynamic code.
This is an extremely clear explanation of Swift making a different set of trade-offs to achieve a different goal, and it's worthy of consideration and evaluation. (I read it when it was first published.)
There are no attacks against Rust here. This is a professional write-up discussing two languages, by someone experienced with both.
This is an extremely accomplished person writing about something she is possibly more qualified to than any other person on earth.
> Also some folks like to complain that Rust doesn't bother with ABI stability, and I think looking at how Swift does helps elucidate why that is.
But in this particular case, you should probably know that the author is not some nobody trying to piggyback on Rust's popularity :) They're a pretty well known contributor, in fact.
However in the big context of application programming, and how Swift is used on Apple stack, any form of garbage collection alongside memory ownership is much more productive.
Long term adoption for Rust will be places like where MISRA-C and SPARK are used nowadays.
If I have to spray my code with Arc, Rc and RefCell, I rather let the compiler type them for me.