The LKM post mentions binary size improvements.
One issues I have had with Rust applications is the huge binary size (yes, I know this has improved a bit lately). Is there a good comparison between kernel C and kernel Rust code in this regard?
One issues I have had with Rust applications is the huge binary size (yes, I know this has improved a bit lately). Is there a good comparison between kernel C and kernel Rust code in this regard?
This talks about going to extreme lengths on making the smallest Rust binary possible, 400 bytes when it was written, https://darkcoding.net/software/a-very-small-rust-binary-ind...
The thing is, you lose a lot of nice features when you do this, like panic unwinding, debug symbols, stdlib… for kernel and some embedded development it’s definitely important, but for most use cases, does it matter?
In ye olden days it was common to distribute a binary without debug symbols, but to keep a copy of them for every released build¹. If an application crashed (panicked, signalled, etc.) you got a core dump that you could debug using the stripped binary together with the symbol file. This gave you both smaller binary sizes and full debugging capability at the cost of some extra administration. I'm not sure if this is possible with "stock" Rust, but if you need lean binaries but want to do forensic investigation it's something to look into.
1. https://sourceware.org/gdb/current/onlinedocs/gdb.html/Separ...
1: https://doc.rust-lang.org/cargo/reference/profiles.html#spli...
At that point I suspect the biggest culprits are overuse of monomorphisation, and often just more stuff happening compared to equivalent C++ code because the language makes larger code bases more maintainable. I'd also count some niceties in that category like better string formatting or panic handling, which is an insignificant cost in any larger software but appears big in tiny hello-world type programs.
This is also true for everything in general. Having the one best thing for foo isn't as helpful as an array of choices, each with different tradeoffs. You simply choose the one best for your needs. Whether it's cheese at the supermarket, an webserver framework, operating system intrinsics, or command line argument handling. Some things can be standardized and serve as a common base for everyone, but it's challenging to do that without at least one person's requirements. Standards also always feature creep until someone tries to reset it with a new standard which is less complex, but I guess that's a different topic.
Do you want a Doodad, a Gooba or a Wumsy? No idea? Me either. So until I care, I'd rather not be asked to choose. But once I discover that I need something with at least 40% Flounce, I can see that Doodads and Goobas both are rated at 50% Flounce, whereas Wumsy has only 10% Flounce, now we're making an informed choice, it should be easy enough to insist on a Doodad to meet my requirement.
If I measure that Monomorphization is out of hand in my codebase I can use dyn to get that back under control for a fair price, but I think the default here is sound.
Sometimes you can provide `T = Arc/Box<dyn Foo>` where `T: Foo` is required, but only if the trait is designed to be object-safe, not simply by default. If you get to design the trait and all of its consumers yourself, you might have this option, but it's very possible that you're using a library that does not make this possible. You can easily be the first person to bother trying the `dyn` for a trait and running into these limitations.
Besides that, you might not even have that much control of the concrete type used. For example, if you are generating large schemas with serde, serde decides how that code is monomorphized, not you. In contrast, for better or worse, the path of least resistance in Go is to use a reflection-based serialization framework which has notable runtime costs (that may or may not matter to a given project) but successfully avoids compile time and binary size costs. (There are other reasons that Go binaries end up even larger than Rust ones, this just isn't one of them)
Despite Rust's general principle of giving its users informed choices here, I am not aware of any option that does 100% dynamic dispatch for (de)serialization, so in practice this is a largely unavoidable cost in each project that is decided only by how complex the schema is.
It's also only fair to point out that C++ tends to end up in this place too, mitigated only by dynamic linking and not any magical property of the language itself. Even C can head this way because monomorphizing with macros has the same effect, though due to how such code is structured, it's also less likely to be inlined than C++ or Rust.
But of course that's an anti-pattern because they are in reality likely to forever just return Vec<T> and knowing that helps you. My early choice only makes sense if either I can't tell you anything more specific than impl IntoIterator<Item = T> or I already know I intend to make a change later. So these days I almost always write down what exactly is returned unless either I can't name it or no reasonable person would care.
For serde in particular my guess is that if you need lots of dynamism serde is the wrong approach even though it's popular. It might be interesting to build a different project which focuses on dynamic dispatch for the same work and tries to re-use as much of the serde eco-system as possible. Not work which attracts me though.
Because I didn't admit you were getting a Vec, if you actually need a Vec you actually can't just use the one I gave you. You must jump though hoops to turn whatever I gave you into a Vec, bloating your code.
The implementation is pretty clever, it is probably not going to meticulously take my Vec to pieces, throw it away and make you a new one, instead just giving the same Vec. But this trick is fragile, so much better not to even need it.
For things like a kernel, it is moot as most deps are simply not possible to use anyway.
When you consider the full ecosystem, you need to really compare it to alternatives in largely managed languages like Java, go, node, etc. those binaries are far larger.
What's an example of this for, say, libcurl? On my system it has a tiny number of recursive dependencies, around a dozen. [0] Furthermore if I want to write a C program that uses libcurl I have to download zero bytes of data ... because it's a shared library that is already installed on my system, since so many programs already use it.
I don't really know the appropriate comparison for Rust. reqwest seems roughly comparable, but it's an HTTP client library, and not a general purpose network client like curl. Obviously curl can do a lot more. Even the list of direct dependencies for reqwest is quite long [1], and it's built on top of another http library [2] that has its own long list of dependencies, a list that includes tokio, no small library itself.
In terms of final binary size, the installed size of the curl package on my system, which includes both the command line tool and development dependencies for libcurl, is 1875.03 KiB.
[0] I'm excluding the dependency on the ca-certificates package, since this only provides the certificate chain for TLS and lots of programs rely on it.
The market and your boss do not care about that. They want tasks X and Y done. You have no time to vet 15 alternatives and pick the most frugal one in terms of binary size. Not to mention that for many tasks you have no more than 3-4 alternatives anyway, and none of them prioritize binary size. What are you going to do? Roll your own? Deadline is looming ever closer, I hope you can live without sleep for several days then.
We all know the ideal theory.
Downloading 3GB of dependencies is not a thing that happens in the Rust ecosystem. Reality is orders of magnitude smaller than that. Why are you exaggerating so much?
Some people bristle at the thought of external dependencies, but if you want to do common tasks it makes sense to pull in common dependencies. That’s life.
Assuming they're talking about the built size of dependencies that are left lying around after cargo builds a binary, they're really not exaggerating by much. I have no difficulty of believing that there are Rust projects that leave 3GB+ of dependency bloat on your file system after you build them.
To take the last Rust project I built, magic-wormhole.rs [1], the source code I downloaded from Github was 1.6 MB. After running `cargo build --release`, the build directory is now 618 MB and there's another 179 MB in ~/.cargo, for a total of 800 MB used.
All this to build a little command line program that sends and receives files over the network over a simple protocol (build size 14 MB). God forbid I build something actually complicated written in Rust, like a text editor.
Also why those GNU/Linux repos are actually multiple DVDs nowadays.
I'm not sure I understand your point with these, as of course no one ever installs the complete repository (e.g. all of Debian), because there's a ton of software in it you don't need or want. Assuming you mean the installation media, at the very least Arch Linux is still less than 1 GB.
Moreover, I think the point in comparing the behavior of Rust dependencies with other ecosystems (C, C++, Haskell, Python) is that most of this cruft is left behind in the individual directories used to build the software. I occasionally write programs to solve some problem, or for fun, and usually I have to download nothing at all, because I can rely on the dependencies supplied by my system and already installed on behalf of other programs (yes, I'm well aware that this doesn't cover all use cases). Rust is fundamentally not designed to work that way, and the large build sizes and huge dependency trees have a multiplying effect on that foundational issue.
Turn off the standard library and your binaries can be incredibly small. This is how it’s used in microcontrollers and the Linux Kernel doesn’t use the full standard library either.
There’s panic_immediate_abort unstable setting that makes Rust panics crash as hard as a C segfault, and only then you can get rid of a good chunk of stdlib.
I’ve used Rust for some embedded side projects and I really wish there was a way to just get some unique identifier that I could translate (using debug symbols) to a filename and line number for a crash. This would sort of be possible if you could get the compiler to put the filenames in a different binary section, as you could then just save the address of the string and strip out the actual strings - but today that’s not possible.
I generally find Rust binaries to be "a few" megabytes if they don't have an async runtime, and a few more if they do. It has never bothered me on an individual program basis, but I can imagine it adding up over an entire distribution with hundreds of individual binaries. I see the very real concern there, but personally I would still not risk ABI hazards just to save on space.
However, most complaints are about size of “Hello World”, which in Rust is due to libstd always having debug info (to be fixed soon), and panic handling code that includes backtrace printing (because print to stdout can fail).
Printing of backtrace is very bloaty, because it parses and decompresses debug info.
FWIW, you can configure this in Cargo.toml:
[profile.release] strip = true
That change will be live on 21st March, so manual strips won't be required after that.
i.e. all programs are larger by 275KB, not larger by 20x.
Rust doesn’t have the privilege of having a system-wide shared stdlib to make hello world executables equally small.
The overhead comes from Rust having more complex type-safe printf, and error handling code for when the print fails. C doesn’t handle the print error, and C doesn’t print stack traces on error. Most of that 200KB Rust overhead is a parser for dwarf debug info to print the stack trace.
linux-vdso.so.1 (0x00007fff25cb8000)
libc.so.6 => /lib64/libc.so.6 (0x00007fe5f08d9000)
/lib64/ld-linux-x86-64.so.2 (0x00007fe5f0ae2000)
linux-vdso.so.1 (0x00007ffc109f9000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f8eda404000)
libc.so.6 => /lib64/libc.so.6 (0x00007f8eda222000)
/lib64/ld-linux-x86-64.so.2 (0x00007f8eda4a8000)
Dynamically linked libs rarely contribute heavily to binary bloat
More generally, you statically link something to avoid distribution hassles of various kinds, not because you care about the specific number.
Can you quantify this? How big is too big? Ideally for a real program, and not an experiment to make the tiniest possible program.
On my Windows machine ripgrep rg.exe is just 4.2mb. Making that smaller feels irrelevant.
I’m not convinced that binary size is a real problem. But I’m open to evidence!
I do still own my first computer, an IBM PS/2 Model 50Z, which still has its original floppy drive. Other parts I upgraded -- the 286 was replaced with a 386 SX/Now!, the 30MB ESDI was upgraded to 100MB, and it now has a full 2MB of RAM. I keep the floppy drive because it reads disks that no other floppy drive has been able to read.