Why is a Rust executable large?
lifthrasiir.github.io
lifthrasiir.github.io
We use CPU profilers to tell us why our programs are slow. My tool is intended to be a size profiler: why is my program big?
It turns out there are some really interesting analyses and visualizations you can perform to answer this question.
A few direct comments on the article:
- I do not recommend stripping your binaries completely! Use "strip --strip-debug" instead (or just don't compile with -g). You should realize that debug information and the symbol table are two separate things. The debug information is much larger (4x or more) and much less essential. If you strip debug information but keep the symbol table, you can still get backtraces.
- I don't believe -Wl,--gc-sections has any effect unless you also compile with -ffunction-sections and/or -fdata-sections, which these examples for C/C++ do not.
If you care about binary size in C or C++ you should probably be compiling with -ffunction-sections/-fdata-sections/-Wl,--gc-sections these days. Sadly, these are not the default.
- `-Wl,--gc-sections` does have an effect even at the absence of `-ffunction-sections` (and so on) because libc and libstdc++ is already compiled in the way allowing for GC---perhaps necessarily. The example had a single function anyway and I was lazy enough to exploit that `-ffunction-sections` wouldn't have a difference...
[1] https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Fil...
It's strange -- I can verify with -Wl,--print-gc-sections that this is indeed discarding some sections from a static glibc link, so I was wrong about that. On the other hand I can also see plenty of sections in libc.a that have more than one function in them -- not sure why this would happen if libc was indeed compiled with -ffunction-sections.
I agree that separate debug files are nice, and OS X's implementation of them is especially nice, since it does a very good job of finding the right debug information for an executable.
IIRC the coding convention of glibc is that most functions (and probably all public functions) are contained in their own files. So it effectively results in the similar effect, even when `-ffunction-sections` is missing.
https://wiki.debian.org/AutomaticDebugPackages
Ubuntu/Fedora has been doing this for years too.
The space savings come nowhere close to the benefit of having the additional information available when debugging.
Some operating systems, for example SmartOS, and any other OS based off of the illumos's source code, even inject the entire source code into the ELF binary and library in a special compact format during the build with the ctfconvert(1ONBLD) / ctfmerge(1ONBLD) tools[1][2].
If you are ever considering stripping the binary just to save some disk space but do not have a good reason for it (like building for a space-constrained appliance), please abstain from doing so; every developer and engineer trying to debug your program will be thankful to you if you do not remove the debugging information.
[1] http://lethargy.org/~jesus/writes/mdb-ctf-dwarf-and-other-an...
[2] http://dtrace.org/blogs/rm/2013/11/14/userland-ctf-in-dtrace...
I've made useful firmware for a micro-controller (yes in Rust) which is just 5KB. You can, as the article shows how, dynamically link and then it's about the same as the equivalent C/C++.
The point is, there's nothing inherent about Rust - the language - which results in binaries appreciably different in size to what you can achieve in C/C++.
That's cool. Any pointers regarding how to get started with that?
Would you mind sharing which microcontroller and how to get a Rust compiler for it? I would love to use Rust to program microcontrollers. Every time I've looked into this, I've thought it wasn't possible, because I don't see any microcontrollers in the list of supported platforms here (https://doc.rust-lang.org/book/getting-started.html#platform...) or here (https://github.com/rust-lang/rust/tree/master/mk/cfg). I've considered trying avr-rust (https://github.com/avr-rust/rust), but the README says "NOTE: This does not currently work due to a bug." Any pointers to get started would be appreciated.
On microcontrollers, you use libcore instead of the full libstd, for lack of an OS or memory allocator. Libcore is easy to cross compile because it has no dependencies. You provide a target JSON file to specify LLVM options, and a C toolchain to use as a linker, and nightly rustc can cross-compile for platforms supported by LLVM.
https://github.com/hackndev/zinc is a good starting point.
Things like interrupt handlers and statically-allocated global data require big chunks of unsafe code. Rust has a promising future in this space, but it will take more experimentation to get the right abstractions to make microcontroller code more idiomatic.
That's an encouraging start.
Having been there, and as reported on weekly basis by numerous people like @internetofshit, much of the code that runs the current IoT hype is utter and complete tripe. I have a faint hope that Rust could be something to help in that regard, even though at the same time I recognize that much of the badness in IoT has to do with financial considerations (too costly to make a good quality talking coffee-maker). Maybe it's especially because IoT seems to be all about getting cheap stuff out cheaply, I have this hope that at least having the language and development ecosystem help devs instead of shooting them in their foots would be a good start.
I was wondering if the Rust front end of llvm can be ported to other compiler engines. That could open up the number microcontrollers supported quite quickly.
$ find /usr/bin -type f | wc -l 2254 $ du -sh /usr/bin 445M /usr/bin
If all these programs were written in Rust and statically compiled, assuming only a 600K difference by binary, that would make my /usr/bin 1.3G (or 300%) larger.
But in reality all those programs are dynamically linked against many libraries in /usr/lib, so the difference would be even bigger, with libraries duplicated between all those programs.
Sure, you can dynamically link with Rust too, but then, you hit the other problem that there is no stable ABI (yet?), and that upgrading Rust (every 6 weeks) means recompiling everything.
You could go the way of busybox or uutils and have a single binary with many hard links. So 'ls', 'wc', 'grep', etc can all point at a single executable which dispatches to different functionality based on argv[0].
Then you can even share code between the binaries, which should make them even smaller.
$ dpkg -L coreutils | grep -c /usr/bin 76
Are you suggesting that all of e.g. GNOME should be a single program?
Dynamic linking was an optimization which came about when memory was expensive. Memory is no longer expensive.
Is 1.3GB (or even 13GB) a lot on your hardware[1]?
According to "Do not prematurely optimize": Pretending we never had dynamic linking, and given today's hardware constraints[2], as a community would we choose to reimplement and universally adopt this optimization?
[1] Keep in mind that a single "modern" application, on average, weighs in the 10s of MBs or GBs.
[2] I'm talking about the general case, for the majority of OS distributions, ignoring the relatively exceptional case of embedded systems, which do in-fact still need it.
"Do not prematurely optimize" is not a software design rule, it's a time-management rule. Dynamic linking has software design impacts.
No, it's both. Optimization often affects the cost of later decisions, and the reason not to prematurely optimize is because it can easily take you to a local optimum which is not very optimal at all. This is a perfect example of that, as the GP comment point out. If memory is not constrained as it was when this trade-off became common, it may not have become prevalent. Static binaries are faster (the degree to which depending on a lot of factors), while dynamic binaries are smaller on disk and in memory, if the shared libraries are already used elsewhere. Modern optimizations at the OS level for forking and threading should make consideration of those negligible.
So here is my point: optimization that can affect the relevant interfaces of your software is not premature because deciding on the interfaces your software exposes is not premature.
Nit:
"Use it for the majority of platforms [and/or for the majority of applications]?"
Even latest desktop processors have a L3 cache of only a few MB.
And sufficiently aggressive inlining will increase the program size further. This might or might not be compensated for by the increase in instruction-pointer locality.
So the arguments against inlining apply even more strongly when talking about every program being statically linked, the same code (standard library) will exist in memory in many places, and will get dumped and reloaded to L2/L3 every process swap. Nothing slower than having to wait for something to be faulted in.
There is something slower. When your executables are so large that you have to hit the swap drive on process swaps.
I am under the impression that when process switches the CPU caches are flushed.
(That's one reason why it's beneficial to schedule a process on the same CPU if possible - the data is still in the cache)
I have no choice but to use DSOs for our Rust code.
At least I hope "methodologies adapted", I don't use docker images, so that's an assumption on my part, but I feel it's a fairly safe bet.
... but since I'm already playing devils advocate :)
Dynamic linking also lets you update libraries ... and cause security issues simultaneously across all applications. Increasing the number of possible attack vectors to successfully utilize that vulnerability.
I've seen this in my day job at Pivotal. The buildpacks team in NYC manages both the "rootfs"[0] of Cloud Foundry containers, as well as the buildpacks that run on them.
When a vulnerability in OpenSSL drops, they have to do two things. First, they release a new rootfs with the patched OpenSSL dynamic library. At this point the Ruby, Python, PHP, Staticfile, Binary and Golang buildpacks will be up to date.
Then they have to build and release a new NodeJS buildpack, because NodeJS statically links to OpenSSL.
Buildpacks can be updated independently of the underlying platform. The practical upshot is that anyone who keeps the NodeJS buildpack installed has a higher administrative burden than someone who uses the other buildpacks. The odds that the rootfs update and NodeJS buldpack are updated out of sync is higher, so security is weakened.
Dynamic linking is A Good Thing For Security.
On my phone or tablet, that is most definitely a lot.
At that point, an extra 1.3GB would just be icing on the download.
I don't really care which point-release of zlib my program is linked with, I just want to decompress stuff. If someone finds a bug (or exploit), I am not the best person to quickly realize it and release an update -- the maintainer of zlib, and the packagers, and OS distributions, and sysadmins are in a much better position. But if it's statically linked, then developers have to be involved.
You could say that we could invent a mechanism to allow sysadmins to rebuild with patched libraries, but then we'd still need to reinvent all of the versioning and other headaches of dynamic libraries.
I think dynamic libraries are kind of like microservices. Sure, they can break stuff, but they allow higher degrees of complexity to still be manageable.
E.g if an author of a program finds a problem in the dynamic library he or she is using, the problem is forced to be solved upstream, benefitting all users of the library. If instead static linking was the norm, it is much more likely that the author would just solve the problem for him or herself and the solution would never reach the wider community.
In the best of worlds, we would have static linking everywhere but the "social contract" of dynamic linking would be enforced just as strongly.
1) The problem was really sending back uninitialised memory. In Rust you can't have uninitialised memory. The oversize allocated buffer would have to have initialisation data passed in (possibly zeroes)
2) You'd never write the Rust code like that anyway. The abstractions avaialble mean that you aren't separating the content of some data and the length to pass to allocators.
That isn't to say that you can't do something similar in C, but it is an order of magnitude more challenging to design a "module" in C that is explicit and robust compared to the effort to do the same in Rust. I've coded my fair share of cryptographic systems in both C and Rust. Bulletproof C is just _exhausting_ to code and work with. The same kind of code in Rust is, dare I say, fun to write. It's just a joy to use Rust's type system to enforce rules and invariants, and then codify those rules in the documentation comments above the structs/functions, and then have "cargo test" actually run the code in that documentation automatically to check it for validity.
And yes, as you point out, some of the big bugs lately have been logic bugs resulting not necessarily from poor code but from poor design. Thing is, the less mental capacity a language requires from a coder the more mental capacity that coder has to use for thinking about the application logic. i.e. in C when you get a string you have to think about how to handle the UTF-8 encoding and what to do about path names that somehow ended up with a non UTF-8 character, and whether the string is NULL terminated or pascal, and is memmove (src, dst), or (dst, src)? In Rust, well, that's all handled, so you think about what the string actually means and, hopefully, you'll realize that hey you should probably sanitize that string so it can't be used to gain shell access from an SVG file.
Phones? IoT? Embedded? OS devs? Someone just checked in a 14KB binary size reduction to our shell (by removing unnecessarily virtual methods in some C++) that was widely celebrated.
Also, smaller .dll's load faster from disk.
$ cat ../example/hello_world/hello.zig
const io = @import("std").io;
pub fn main(args: [][]u8) -> %void {
%%io.stdout.printf("Hello, world!\n");
}
$ ./zig build ../example/hello_world/hello.zig --export exe --name hello --release --strip
$ wc -c hello
5760 hello
$ ldd ./hello
not a dynamic executable
$ ./hello
Hello, world!
Also note that Rust is a lot further along than Zig right now. Zig does not have backtraces or threads yet. But I believe that the executable size for hello world in release mode will not contain backtrace code, or threads, or a memory allocator, even when Zig catches up to Rust in terms of std lib functionality.
[1]: http://ziglang.org/
LTO is much older than this, even for C/C++
Just not in GCC :)
package main
func main() {
println("Hello, world")
}
Although this would be ill-advised, kind of like PHP in general..
So far it only works on GNU/Linux I think.
# perl -E 'my $cmd = shift; use Time::HiRes; my @times; for (1..10) { my $start=Time::HiRes::time; my $out = system($cmd); my $stop = Time::HiRes::time; die if $out>>8; my $time = $stop - $start; push @times, $time; printf "%0.4f\n", $time; } @times = sort {$a<=>$b} @times; @times = @times[1..8]; my $cumulative=0; $cumulative += $_ for @times; my $average = $cumulative/8; printf "Average time of 10 runs of \"%s\", dropping best and worst: %0.4f\n", $cmd, $average;' "perl -e '1;'"
0.0039
0.0032
0.0031
0.0031
0.0032
0.0035
0.0036
0.0030
0.0033
0.0032
Average time of 10 runs of "perl -e '1;'", dropping best and worst: 0.0033
A minimal C program (just return success from main after including stdlib and stdio)
You do realize that a "including stdlib and stdio" means nothing for a C program, right? These are, literally, just the API definitions, in the Oracle-vs-Google Android sense. The default gcc options probably produced a dynamic executable; if you compile it statically, you might be able to shave a few more microseconds in startup time.
The only difference from the shell builtin time, or /usr/bin/time, should be the shell startup and exec call. Every command is fork+exec, so you can't really get away from that. If I really cared, I would try to account for that, but I don't. I thought it was pretty obvious that I wasn't being very rigorous. I just did the minimum to make the values I got not useless.
> which on my machine is reliably 0.002 real seconds, ⅔ of your times
And on mine it fluctuates between 0.002 and 0.009. You mention your times relative to my times, but that's useless. I ran mine on a 512mb VPS. The only relevant measure to determine overhead would be my method vs the shell builtin. Is that what you were actually referring to?
> You do realize that a "including stdlib and stdio" means nothing for a C program, right?
Truthfully, so I wouldn't have to remember the setup for C, I just googled "minimal C program" and removed the printf it had. I only mentioned the includes for completeness sake, and I didn't want to clutter the comment with the source.
> if you compile it statically, you might be able to shave a few more microseconds in startup time
Possibly, but that's not really what I was trying to convey. I was just pointing out that a minimal Perl program isn't slower than a minimal C program, and it's notable if you can get your startup times close to that for a system with a virtual machine.
At that moment, I think most language developers should turn to LLVM to spread their languages more rapidly, rather than trying to mess around with all the realities of language making, especially when it involves statically typed and compiled languages.
$ cargo rustc --release -- -C prefer-dynamic
$ ls -al target/release/hello
-rwxrwxr-x 1 lifthrasiir 8831 May 31 21:10 target/release/hello*