Hacker News Reader

Comment by clessg | Hacker News Reader

How are compile times now? The one barrier I had to using Rust for anything large was the awful compile times.

Very happy to see the tooling improvements, though. Great work!

Well, "now" is relative. When did you last try? They've been steadily dropping each release, though this latest one isn't as large as the last few.

Apologies, I should have mentioned that. It's been about 9 months.

Since then Rust 1.2 had a big focus on performance: http://blog.rust-lang.org/2015/08/06/Rust-1.2.html

I measured ~40% improvement from 2015-05-10 to 2015-10-30. That is, it takes 0.6x time to compile. 2015-05-10 was chosen because Rust 1.0 was released 2015-05-15.

https://github.com/sanxiyn/sandbox/blob/master/rust/time-2.p...

I'm not sure what's considered slow or fast, but I just tried a couple of larger-ish rust projects. All data on a midrange SSD, i7 ~4.4Ghz. Warm disk-cache, so pratcically everything in RAM.

  crates.io backend service:
  # Note the thing pulls down lots of dependencies doing
  # install, so this is after a "cargo build;cargo clean":

  git clone git@github.com:rust-lang/crates.io.git crates.io.git
  cd crates.io.git/
  ./script/init-local-index.sh 
  multirust run nightly cargo build --release
  multirust run nightly cargo clean
  # 4 threads, ~100% CPU up to compiling the final cargo.io bit,
  # which takes the longest:
  git diff
  diff --git a/Cargo.toml b/Cargo.toml
  index cf99b60..02e54fb 100644
  --- a/Cargo.toml
  +++ b/Cargo.toml
  @@ -5,6 +5,7 @@ version = "0.1.0"
     
  [profile.release]
  opt-level = 2
  +codegen-units = 4
     
  [lib]
  name = "cargo_registry"

  $ time multirust run nightly cargo build --release
  real    1m29.082s
  user    4m47.276s
  sys     0m6.196s

  # Single threades (default Cargo.toml):
  real    1m41.210s
  user    4m0.376s
  sys     0m5.120s

Servo:

  git clone https://github.com/servo/servo servo.git
  cd servo.git/
  ./mach build --release

(Servo also pulls down lots of dependencies, including a full rust toolchain...)

  ./mach clean
  time ./mach build --release
  Build completed in 791.07s
  real    13m11.874s
  user    40m21.468s
  sys     0m51.304s

At approximately 100% cpu for the first part, either the servo build script makes sure things are parallelized, or the Cargo.toml-settings are set for parallell builds (haven't checked).

And finally, a smaller project:

  git clone https://github.com/Aaronepower/tokei.git
  cd tokei/
  cargo build --release
  cargo clean
  
  $ time cargo build --release
  real    0m30.354s
  user    0m32.708s
  sys     0m0.444s

According to tokei (which is a blazingly fast cloc tool), there's ~100k cloc of rust in servo, ~7500 cloc of Rust in crates.io (but that doesn't include all the dependencies...).

After doing a clean install of rust stable with multirust, the combined cloc of stuff under ~/.multirust...cargo/src and tokei is ~11.5k cloc of rust, along with ~ 1400 cloc of c++.

You should note that "cargo clean" causes dependencies to be rebuilt next build as well; a regular "cargo build" would only re-compile your module and re-link with already-built dependencies, and so should be a fair bit faster.

Good point. I did that on purpose in order to do a full rebuild -- but I suppose for the purposes of "is rustc fast", it's actually more interesting to just do a build without clean (although one would have to manually remove the built binary and/or change the code in order to force a rebuild?).

I'm particularly interested in incremental recompilation. From what I can see it hasn't landed yet (?) - but I can't seem to find any ETA on when it will.

There isn't a strong ETA, no. Everything about internals is basically waiting until after the HIR/MIR work lands, as it would end up changing a lot anyway, so that's where the focus is at the moment.

That doesn't mean the _discussion_ about how it works has to stop in the meantime, though...

Well even a guesstimate would be nice :)

I've been following Rust from sidelines before 1.0 and from my perspective there are advantages but I can also see a lot of pain in porting stuff and moving over and I right now the benefits don't outweigh the risk and cost for me - but having a fast build system (which implies incremental recompilation) along with a default package manager (which it already has) would be enough for me to start working with it seriously.

It's kind of sad that building code is such an instrumental part of developing C++ when it's just about dealing with technical issues of the compiler/language/tools and not the problems I'm trying to solve - anything that gets me away from that while leaving me with similar level of memory control and portability is a big win for me.

So when should I be looking back at it ? 6 months ?

If you want a guess, I guess "in 2016".

There's two components to this. I just asked the relevant people, they said "Six months is a safe guess, we hope sooner, we have ambitious plans, we'll see."

However, there's also something mentioned in this announcement: cargo-check. In terms of the "I'm building a project and I don't want my builds to be slow," I think that cargo-check might even be more important. cargo-check relies on an important insight: many times, when you're compiling, you don't actually care about producing a binary, you care that your code passes typechecking and other passes. cargo-check basically runs those things, and doesn't produce the final binary. Makes sense?

Is this something which the editor plugins can make use of? (as a quick lint/feedback about the code). My guess is that the plugins actually use naked rustc call rather than cargo build.

Currently, it basically runs `rust -Zverbose -Zno-trans`, so yes. If it gets more advanced, we'll see.

What would make you choose swift vs rust at this point? The compile speeds have been getting pretty close to objective-c levels, there is 1st and 3rd party package management and its been recently open sourced and is actually being developed in the open.

On top of that they have the financial backing of apple.

Does Swift run on Windows ? Emscripten ? Android ? Portability is also a big question that Rust has a lot better incentives to solve than Apple.

I got the impression that Linux support is more of a "here iOS/OSX develoeprs - we're aware you don't actually run OSX on your servers so you can share code between client and server now". I was not aware Apple wants to push Swift as a cross platform language.

If it was and their commitment looked credible (ie. they start porting tools like integrating debuggers and stuff on other platforms as well) I would definitely consider it.

For the ignorant among us, do you have a link explaining what HIR/MIR is (and how it relates to Rust)?

I should blog about it, because I keep writing comments. :)

So, the basic process of compiling is:

  source -> AST -> LLVM IR -> asm

At each step, you can do transforms too, so like, LLVM will take in LLVM IR, but before compiling, will simplify/transform it into other IR.

There's a few issues with this. The first is that any processing we want to do, like optimizations, safety checks, etc, has to work on the AST. This means that when the AST changes, these features need to change. This is why compiler plugins aren't stable, we're not ready to stabilize the AST yet. Doing so would mean things like "we can never add a new keyword", or at least, makes that process harder.

The final process for Rust will look like

  source -> AST -> HIR -> MIR -> LLVM IR -> asm

HIR is "higher IR" and MIR is "mid IR". In this sense, LLVM IR is a "low IR". There's two important aspects here: the first is that this decouples the AST from things like plugins, which can operate on the HIR, which we will stabilize. The second is that at each step, things get simpler. Here's an example. We have both regular old if statements, and match, which can do more powerful, structural things. Both are fundamentally jump statements, though, and so we can transform the more complex match into the more primitive if. This means that by writing safety checks, optimizations, and other things against MIR, they're both easier to write, as well as more maintainable.

There's one other minor benefit: Someone who wants to write an alternate compiler could theoretically write MIR -> ASM, and eliminate LLVM entirely. While we have less than zero percent chance of doing that ourselves, it might be nice if you want to support some sort of esoteric platform LLVM doesn't, for example.

Does that all make sense?

It does make sense, and thank you very much for taking the time to explain! :)

What does HIR/MIR mean ?

https://news.ycombinator.com/item?id=10713301

How long is "long"? What sort of time would we be talking about for 100KLOC projects?

I don't know about 100K, but for my 5667 line project, a recent nightly build takes 8 seconds to build on my 2012 macbook pro (4 core 2.3G i7). That is with up-to-date external crates that aren't being rebuilt.

Typecheck happens early, with borrow-checking following soon after, so you get feedback on errors fast, and thus only end up waiting when the compiler is happy with your code. Still, it may be an issue for people who like to do TDD or need to do a lot of run-time verification.

For a big project you'd probably need to split it up into multiple crates to "emulate" incremental compilation, at least until it is supported by the build system.

That exact property is why cargo check works.

You would definitely split that project up into multiple crates at the moment. 100KLOC would take an eternity to compile. My guess would be 15 to 45 minutes for a release build, depending on things like how much you were using generics.

Reply on news.ycombinator.com