Hacker News Reader

Comment by ajross | Hacker News Reader

This gets confused right off the bat between file size and mapped size. The whole "strip" nonsense could have been skipped if the author looked at .text/.data/.rodata (and .bss too -- often the resource you're trying to conserve is RAM and not storage!) sizes instead.

Likewise the playing with upx isn't really relevant to a rust article per se. That's a generic executable compressor that will work with anything.

But some stuff here was really interesting. I honestly had no idea that rust was statically linking its own heap implementation into every binary it created! Come on, guys, jemalloc isn't even in rust and doesn't benefit from static linkage. If you're going to distribute your own runtime (and if you're using your own heap, you're distributing your own runtime!) at least distribute it in a shared-by-default way.

View on HN

> Come on, guys, jemalloc isn't even in rust and doesn't benefit from static linkage. If you're going to distribute your own runtime (and if you're using your own heap, you're distributing your own runtime!) at least distribute it in a shared-by-default way.

Our users consistently indicate that they prefer the convenience of being able to copy all-in-one binaries from system to system to the space savings of dynamic linking. Rust used to dynamically link by default, but based on popular demand this was changed prior to 1.0.

Look at what people say they like about Go, for example. Static linking for easy deployment is high on the list.

http://harmful.cat-v.org/software/dynamic-linking/

I don't agree with this. Dynamic linking is perfectly reasonable for OS libraries like kernel32.dll, for instance. Dynamic vs. static linking is a tradeoff like any other.

Sure, but NT is a totally different story since you just aren't given another stable ABI other than linking against .dlls.

But there is a lot of value on Unixen of static linking (or only linking against VDSO equivalents).

Dynamic linking is indeed harmful, but static linking also in its own ways. As often, there is no silver bullet, just trade-offs.

One of the things that I learned from Google is how much breaks on a machine that has somehow corrupted memory in a dynamically linked library. Every application that you deploy using that library is broken. By contrast if you deploy a statically linked application, it probably works.

If you deploy to enough machines in the cloud, you'll eventually hit this use case.

If you deploy enough machines that corrupted memory becomes a common problem you should look at your healthchecks instead of static linking.

You need to do both.

Large distributed jobs at Google can literally take several times the expected lifetime of an average server to run. When you run such a job, you must expect that it will wind up running on computers of all stages of repair, and some of the computers that it runs on will come out of commission during your job and will have to be replaced by others.

This scale requires defense in depth. Health checks are necessary, but not sufficient. You also need to minimize the problems if the machine you're on is not yet known to be bad. And deal with double checking results because at that scale you can't trust the hardware. And so on.

It was popular demand because tooling was not mature. Now we want to install programs, which are written in Rust, using system package manager, thus we demand dynamic linking.

Moreover, when part of program is static and part is dynamically linked to a C library, it does not matter any more.

For embedded systems, dynamic linking is better when more than one running application is written in Rust, to save memory and processor cache.

Dynamic linking is fully supported by Rust. Just pass -C prefer-dynamic in your RUSTFLAGS.

But ABI is not fixed yet, so all libraries and binary must be compiled by same compiler with same options.

Since the parent mentioned installing using a package manager, could it be default in that case as well as in the `cargo install` case where cargo essentially controls the binary?

If not default, could it be more easily chosen with a shorter option?

Is a rust heap/runtime packaged for distros like Debian which have packaged binaries of some crates?

Note, I'm not in favor of making dynamic default for `cargo build`.

Hey ajross,

I originally included the output of `size` for each step, but I thought it made the text output too large. However in this case, the actual size of binary on "disk" is important, as we were trying to fit this binary on a small embedded linux device, so if we had shipped all of that debuginfo, it would have taken up useful space on the device (which has a total storage size of 8MB).

Regarding UPX, yup, not Rust specific at all, but still relevant when talking about producing tiny binaries.

Re: the allocator, steveklabnik wrote a good response, though I would add that just because something isn't written in Rust, doesn't mean it can't still be well tested, performant code. Why rewrite it in Rust if you don't have to? :)

Anyway thanks for reading and thanks for the feedback!

> I would add that just because something isn't written in Rust, doesn't mean it can't still be well tested, performant code.

I know. My point was that unlike rust binaries (which can benefit from significant static optimization) jemalloc is just a blob and doesn't need to be included in every binary.

It doesn't need to, but our users (including me) want it to, because it makes binaries easier to copy around.

I think the point here is that if you are building a few binaries for a system that has severe storage constraints, shared libraries might be a way to reduce disk usage.

If the library is stable and installed system-wide, it shouldn't be much harder to copy binaries around, and it makes each of them smaller.

Of course, there is also the dual, busybox-way of embedding everything into a single binary. This should lead to even more gains, but moves complexity elsewhere: calling the binaries, making sure no symbols conflicts, more complex compilation chain, etc. Even better gains can be achieved if every tool is developed simultaneously, and is reusing utility functions from other parts of the project (which is actually what shared libraries do, to a lesser extent).

We are interested in switching the default back to the system allocator, but since swapping in your own isn't stable yet, it would be a regression to not be able to include jemalloc instead. That work is underway, and when it's stable, we're likely to change the default, for exactly this kind of reason.

No you've got confused because you think he's trying to conserve RAM but it's very clear in the article that he's targeting low disk size.

> however with their limited flash storage space (8MB for the whole firmware

Shameless plug: my project Bloaty (https://github.com/google/bloaty) is a size profiler that shows you both mapped size and file size out-of-the-box.

Reply on news.ycombinator.com