Hacker News Reader

Comment by kungito | Hacker News Reader

I'd say it's pretty unfair to take clean build time to consider doing something useful. 98% of my rust builds are incremental builds which take 10-15 seconds for a project I have been working on for 9 months full time and I think my dependency tree has easily 1000 dependencies.

View on HN

True, it's not a real problem in most scenarios if incremental compile times are good. I still feel uneasy depending on so many other crates, but this seems to be a level of paranoia that others in the community don't share.

Having 1000 dependencies sounds crazy to me! If there's a bug in even one of them that affects you then there's gonna be a lot of digging to figure out the cause, and possibly multiple pull requests to get it fixed. I think my CPU would spend a bit longer than 10-15 secs on the linker step, too.

Compile/link times are definitely a discussion topic. Here's some quick tips:

  * Incremental compilation helps locally, not so much in CI, unless you save/restore the whole cache which gets large quickly and evicting the out-of-date objects is non-trivial.
  * In CI, sccache helps, but not as much as I'd like: there's a bunch of things that are non-cacheable, notably crates that pull in & compile C/C++ code (I'd trade 2 c-bindings crates against 12 Rust-only crates any day for that reason alone)
  * Splitting stuff across different crates helps parallelizing the build (and caching it better), which may be one of the reasons some projects end up having "1000 dependencies" (although I've rarely seen upwards of 600)
  * For incremental builds, linking does become the bottleneck. Full LTO is especially slow, Thin LTO is better. Switching to LLD improves thing. I'm hopeful that mold will improve things some more.
  * Re multiple PRs: thankfully crate families tend to live in the same repository, so fixing something across both tracing / tracing-subscriber could be a single PR, for example.

I wish the Rust community invested more in build caching, but even with the current state of things, there's often steps you can take to make things better.

Have you tried sccache? https://github.com/mozilla/sccache

It's his second bullet point

Can you combine ccache with a sccache to make the C/C++ parts fast too?

the reason for

> I still feel uneasy depending on so many other crates, but this seems to be a level of paranoia that others in the community don't share. Having 1000 dependencies sounds crazy to me! If there's a bug in even one of them that affects you then there's gonna be a lot of digging to figure out the cause

is that there is far, far more likely to be a bug in the version you write on your own to achieve the same goal than there is in a widely-observed library written by somebody who's chosen to specialize in that specific thing

In theory yes. But in practice that isn't always true. People often don't audit other modules on the assumption someone else had. Which means nobody ends up doing it. And if you end up with an ecosystem that favours more modules over fewer, you can end up with more modules than a given developer or team are willing to audit (a bit like "alarm fatigue" where if you have too many objects to check then people will inevitably just get lazy).

Just look at how many C and C++ libraries are maintained by 1 individual and have almost no 3rd party oversight to see that Rust can't automatically make the claim you made.

That all said, for anything complicated and/or directly security related, one should always check if there is a module first.

I look at it the other way around. You own any bug in your product whether it comes from a dependency or from code of your own; you have to fix the bug either way. Using a dependency doesn’t reduce your responsibility, but it does reduce the amount of code that you have to write yourself.

But if you are willing to own that responsibility then you should read the code you're importing to begin with. I know I do but I also know most people don't bother.

I do acknowledge that there will always be bugs that are identified by your users but equally if you're not auditing your dependencies first then it's hard to argue that you're not just passing off that responsibility wholesale to your users.

It's always a tradeoff whether you want to read some other code or work on something else. Rust ecosystem is not that mature so for a few libraries I had to end up rewriting the thing myself with some fixes or without some bloat. I'm writing an application level thing and I need as many utilities as possible as I do not want to write all the layers for all the abstractions that end up in my product. Then when something breaks I investigate, offer a fix, open an issue or whatever. I'm not writing something that requires too much reliability or whatever, the utility is elsewhere.

> there is far, far more likely to be a bug in the version you write on your own

In general I agree with this, but there is another relevant aspect to consider: something that I've written on my own for a particular project is also likely to be more purpose-built, and therefore simpler.

I rather have 1000 dependencies than re-implement all those 1000 dependencies myself, chase all the bugs that have already been ironed out in those, etc.

My time is worth more than my computer's time.

> Having 1000 dependencies sounds crazy to me!

I was thinking that too, but then I did a quick check of a few of my Java services for work, and many of them have more than 400 dependencies (most transitive, of course), with a few getting up to 600.

Obviously comparing Java and Rust is not exactly apples to apples, but I rarely care much about the number of dependencies in my Java projects, so I guess I shouldn't be too worried about Rust projects either (aside from link time, I guess, which javac doesn't have to do).

Reply on news.ycombinator.com