Hacker News Reader

Comment by glandium | Hacker News Reader

> Just to be cautious: Is it a valid question to ask after all? We have hundreds of gigabytes of storage, if not some terabytes, and people should be using decent ISPs nowadays, so the binary size should not be a concern, right?

$ find /usr/bin -type f | wc -l 2254 $ du -sh /usr/bin 445M /usr/bin

If all these programs were written in Rust and statically compiled, assuming only a 600K difference by binary, that would make my /usr/bin 1.3G (or 300%) larger.

But in reality all those programs are dynamically linked against many libraries in /usr/lib, so the difference would be even bigger, with libraries duplicated between all those programs.

Sure, you can dynamically link with Rust too, but then, you hit the other problem that there is no stable ABI (yet?), and that upgrading Rust (every 6 weeks) means recompiling everything.

View on HN

I would gladly sacrifice that disk space in exchange for system utilities written in Rust. We're still finding vulnerabilities in core utilities, even after all these decades.

Most recent vulnerabilities in core utilities really don't have a lot to do with memory safety though - Shellshock & Imagemagick were input sanitization, other common ones though are still injection vulnerabilities or authentication weaknesses. Heartbleed excluded most major vulnerabilities these days aren't related to memory safety.

Heartbleed is not a real memory safety bug when program reads beyond allocated memory. It is more of improper reuse of previously allocated buffer and could exist in safe Rust just as well.

You're right, there isn't a classic simple buffer overrun that Rust would trivially catch, but you're missing two things:

1) The problem was really sending back uninitialised memory. In Rust you can't have uninitialised memory. The oversize allocated buffer would have to have initialisation data passed in (possibly zeroes)

2) You'd never write the Rust code like that anyway. The abstractions avaialble mean that you aren't separating the content of some data and the length to pass to allocators.

Well, there are for example all those libbfd-related issues that caused running strings(!) on unstrusted files to be unsafe.

Sure, but Rust isn't just about dealing with memory safety. The language also lends itself well to solving other common mistakes by virtue of its design and by being built on modern principals. Idiomatic C promotes throwing around pointers/arrays and hoping that the next coder that comes along to consume a struct reads the docs/header and understands how the data in that struct is supposed to be used. Idiomatic Rust uses its type system to strictly enforce how a struct and its data can/should be used. It's a world of difference and results in drastically less bugs. Not to mention the rest of the Rust ecosystem works in harmony with the language to further reduce bugs. Testing as a first class citizen of the language and its tooling is one of the big ones.

That isn't to say that you can't do something similar in C, but it is an order of magnitude more challenging to design a "module" in C that is explicit and robust compared to the effort to do the same in Rust. I've coded my fair share of cryptographic systems in both C and Rust. Bulletproof C is just _exhausting_ to code and work with. The same kind of code in Rust is, dare I say, fun to write. It's just a joy to use Rust's type system to enforce rules and invariants, and then codify those rules in the documentation comments above the structs/functions, and then have "cargo test" actually run the code in that documentation automatically to check it for validity.

And yes, as you point out, some of the big bugs lately have been logic bugs resulting not necessarily from poor code but from poor design. Thing is, the less mental capacity a language requires from a coder the more mental capacity that coder has to use for thinking about the application logic. i.e. in C when you get a string you have to think about how to handle the UTF-8 encoding and what to do about path names that somehow ended up with a non UTF-8 character, and whether the string is NULL terminated or pascal, and is memmove (src, dst), or (dst, src)? In Rust, well, that's all handled, so you think about what the string actually means and, hopefully, you'll realize that hey you should probably sanitize that string so it can't be used to gain shell access from an SVG file.

Forgetting about Rust for a second. Talking about dynamically linked libs in general.

Dynamic linking was an optimization which came about when memory was expensive. Memory is no longer expensive.

Is 1.3GB (or even 13GB) a lot on your hardware[1]?

According to "Do not prematurely optimize": Pretending we never had dynamic linking, and given today's hardware constraints[2], as a community would we choose to reimplement and universally adopt this optimization?

[1] Keep in mind that a single "modern" application, on average, weighs in the 10s of MBs or GBs.

[2] I'm talking about the general case, for the majority of OS distributions, ignoring the relatively exceptional case of embedded systems, which do in-fact still need it.

Yes, we would. Not because it's 'premature optimization', but because it's easy optimization.

"Do not prematurely optimize" is not a software design rule, it's a time-management rule. Dynamic linking has software design impacts.

> "Do not prematurely optimize" is not a software design rule, it's a time-management rule.

No, it's both. Optimization often affects the cost of later decisions, and the reason not to prematurely optimize is because it can easily take you to a local optimum which is not very optimal at all. This is a perfect example of that, as the GP comment point out. If memory is not constrained as it was when this trade-off became common, it may not have become prevalent. Static binaries are faster (the degree to which depending on a lot of factors), while dynamic binaries are smaller on disk and in memory, if the shared libraries are already used elsewhere. Modern optimizations at the OS level for forking and threading should make consideration of those negligible.

Dynamic linking wasn't invented as a premature optimization though. And if it didn't exist today, it would still not be premature to invent it, because dynamic linking does not only concern how large and fast your program is, but also how it is interacted with.

So here is my point: optimization that can affect the relevant interfaces of your software is not premature because deciding on the interfaces your software exposes is not premature.

You are choosing to focus on the "premature optimization" wording, which is fair, it was said. I'm focusing on "as a community would we choose to reimplement and universally adopt this optimization?" (emphasis mine). I think it would be implemented, I do not know of any evidence that makes me believe it would become universally adopted given modern resources.

What do you mean by 'universally adopted?' Available on all platforms? Default linking strategy? All code is dynamically linked?

I'm not sure exactly what was originally meant, I interpreted it as how currently dynamic linking is the norm, is used in every mainstream OS and most the applications run on them, and all the major mobile platforms. If we had to make a choice right now without the history of dynamic linking behind us, would we still choose to use it for the majority of platforms?

FWIW: You appropriately articulated how I meant 'universally adopted'.

Nit:

"Use it for the majority of platforms [and/or for the majority of applications]?"

Is it really easy? It brings with it many versioning problems.

Main memory is cheap but slow. Having a frequently called function in a shared library vs. statically linked code could mean the difference of the code executing from CPU's cache or from main memory.

Even latest desktop processors have a L3 cache of only a few MB.

But static linking could mean that function will be inlined.

It means that function might be inlined.

And sufficiently aggressive inlining will increase the program size further. This might or might not be compensated for by the increase in instruction-pointer locality.

Inlining saves time lost due to jumping about, but it can cost time if it causes code replication (same as loop unrolling), because it can bloat the hot code to larger than the smallest cache.

So the arguments against inlining apply even more strongly when talking about every program being statically linked, the same code (standard library) will exist in memory in many places, and will get dumped and reloaded to L2/L3 every process swap. Nothing slower than having to wait for something to be faulted in.

> Nothing slower than having to wait for something to be faulted in.

There is something slower. When your executables are so large that you have to hit the swap drive on process swaps.

> Having a frequently called function in a shared library vs. statically linked code could mean the difference of the code executing from CPU's cache or from main memory.

I am under the impression that when process switches the CPU caches are flushed.

No. Only TLBs are flushed (and probably only partially). TLBs are used to associated virtual addresses with physical addresses and memory maps are different per processes.

(That's one reason why it's beneficial to schedule a process on the same CPU if possible - the data is still in the cache)

Memory is terribly expensive and I have to fight all of the other developers/product folks/upper management for every byte in my environment (hundreds of thousands of servers).

I have no choice but to use DSOs for our Rust code.

Dynamic linking also lets you update libraries due to things like security issues, it's not just a memory thing. Kinda agree on the space thing too (plus much less chance for things like buffer overflows..)

This was a much more powerful reason before things like docker became common, and methodologies adapted to provide updates for docker images, which for this purpose are functionally identical to a static binary.

At least I hope "methodologies adapted", I don't use docker images, so that's an assumption on my part, but I feel it's a fairly safe bet.

Docker images don't have a nice way of updating without "rebuilding everything". There's a tool called zypper-docker that does allow you to update images, but there's no underlying support for rebasing (updating) in Docker. I was working on something like that for a while, but it's non-trivial to make it work properly.

Hmm, I assumed it would be something along the lines of the images being fairly static, and updated as a whole, and you just apply your configs and data, possibly through mount points.

I was responding to the comment that security updates to libraries make it harder to update static binaries. Docker has revived the problem, and there isn't a way of nicely updating images without rebuilding them (which in turn means you have to do a rollout of the new images). While it's not a big deal, it causes some issues that could be avoided.

Yes, but presumably you're running far fewer docker images than you have binaries that would be affected if you statically compiled everything. For example, I assume in a statically compiled system, an update to zlib will likely affect a lot more packages than docker images you are running (on a server I admin, there's 3 binaries in /bin that link to zlib, and 374 binaries in /usr/bin, which will condense down to some smaller, but still likely quite large set of OS packages). It's easier in a dynamically linked system, where you can just replace the library, but it's not that much better for the sysadmin, as if you want to make sure you are running the new code, you need to identify any running programs that have linked to zlib and restart them, as they still have the old code resident in memory.

this makes the false assumption that updating dynamic libraries never introduces any new bugs.

No, it makes a trade-off. Especially on the stable channel of Debian/RHEL/SLES, an update will most of the time fix more bugs than it introduces.

FWIW: I think everything has its place, and everything has tradeoffs. I can definitely see a lot of usefulness for dynamic linking. The point you raise probably being the best current reason.

... but since I'm already playing devils advocate :)

Dynamic linking also lets you update libraries ... and cause security issues simultaneously across all applications. Increasing the number of possible attack vectors to successfully utilize that vulnerability.

Actually, it's a wash. If all we had was static linking, people would statically link the same common libraries. So you'd have to update multiple binaries for a single vulnerability.

I've seen this in my day job at Pivotal. The buildpacks team in NYC manages both the "rootfs"[0] of Cloud Foundry containers, as well as the buildpacks that run on them.

When a vulnerability in OpenSSL drops, they have to do two things. First, they release a new rootfs with the patched OpenSSL dynamic library. At this point the Ruby, Python, PHP, Staticfile, Binary and Golang buildpacks will be up to date.

Then they have to build and release a new NodeJS buildpack, because NodeJS statically links to OpenSSL.

Buildpacks can be updated independently of the underlying platform. The practical upshot is that anyone who keeps the NodeJS buildpack installed has a higher administrative burden than someone who uses the other buildpacks. The odds that the rootfs update and NodeJS buldpack are updated out of sync is higher, so security is weakened.

Dynamic linking is A Good Thing For Security.

[0] https://github.com/cloudfoundry/stacks

[1] https://github.com/cloudfoundry/nodejs-buildpack

> Is 1.3GB (or even 13GB) a lot on your hardware

On my phone or tablet, that is most definitely a lot.

A library update would require rebuilding every application that uses the library. A change to libc would require effectively reinstalling the OS.

At that point, an extra 1.3GB would just be icing on the download.

Dynamic linking offers modularity and separation of concerns.

I don't really care which point-release of zlib my program is linked with, I just want to decompress stuff. If someone finds a bug (or exploit), I am not the best person to quickly realize it and release an update -- the maintainer of zlib, and the packagers, and OS distributions, and sysadmins are in a much better position. But if it's statically linked, then developers have to be involved.

You could say that we could invent a mechanism to allow sysadmins to rebuild with patched libraries, but then we'd still need to reinvent all of the versioning and other headaches of dynamic libraries.

I think dynamic libraries are kind of like microservices. Sure, they can break stuff, but they allow higher degrees of complexity to still be manageable.

Dynamic linking certainly has no technical advantages like lower memory/disk usage or faster processing. Its main advantage, which has been cited before, is that it forces cohesion in the Linux community.

E.g if an author of a program finds a problem in the dynamic library he or she is using, the problem is forced to be solved upstream, benefitting all users of the library. If instead static linking was the norm, it is much more likely that the author would just solve the problem for him or herself and the solution would never reach the wider community.

In the best of worlds, we would have static linking everywhere but the "social contract" of dynamic linking would be enforced just as strongly.

Not necessarily.

You could go the way of busybox or uutils and have a single binary with many hard links. So 'ls', 'wc', 'grep', etc can all point at a single executable which dispatches to different functionality based on argv[0].

Then you can even share code between the binaries, which should make them even smaller.

ls and grep are in /bin on my system. wc, coming from coreutils is in /usr/bin. But that's really only a small part of /usr/bin.

$ dpkg -L coreutils | grep -c /usr/bin 76

Are you suggesting that all of e.g. GNOME should be a single program?

Why hard links over symlinks?

So someone doesn't unlink the main binary when they are cutting unused utils to save space.

Reply on news.ycombinator.com