Hacker News Reader

Comment by howdydoo | Hacker News Reader

I once had a similar project to OP that took 30 seconds to compile a one-line change (albeit on a 10-year-old CPU). I split it into about 7 crates, and got compiles down to 3 seconds. Since then I'm always vigilant about keeping crates nice and small. I think as long as you're keeping an eye on your crate sizes, compile times won't get away from you.

This is different from C++ where each individual source file can be compiled in parallel, so it's something I had to re-learn.

View on HN

Can this be automated? Could a script divide code into small crates upon compilation?

I did it by hand. It's a mostly mechanical transformation, so in theory it could be automated, but I've never seen an automated refactor for any language that can e.g. take a giant file and decide how to split it up into 5 smaller files. But that's something I'd love to see exist

That's a pretty bad incentive though and you end up with very large dependency trees similar to JS. And we know how that went.

OTOH there might be a benefit too as more code becomes re-usable in independent crates.

> That's a pretty bad incentive though and you end up with very large dependency trees similar to JS. And we know how that went.

That's... not the same thing at all.

Rust lets you have multiple crate in a single workspace, which allows parallelising the build (because crates are the concurrency unit of building rust).

That's got nothing whatsoever to do with pulling random crates, which is a separate issue.

Why doesn't Rust default to one crate per source file then? Wouldn't that make the build massively parallelized :-)

Because you can have circular dependencies within a compilation unit, but not between units.

That actually sounds more appealing to me. In fact it's exactly how OCaml works. Of course in OCaml compilation units are just files already.

I don't think there is a moral justification for circular dependency scopes to always be individual files, especially given that import statements count as circular. The same issue in other languages where circular scopes are individual files leads to some horrible workarounds:

* Python TYPE_CHECKING https://adamj.eu/tech/2021/05/13/python-type-hints-how-to-fi...

* C/C++ header files

But regardless, there is too much Rust code in existence to change this now.

A 'moral' justification is not needed. We are talking about potentially massive build speed improvement here. Sounds like it could just be a build setting.

It would definitely be interesting to run some experiments with automatically splitting up Rust crates into compilation units if possible.

Username checks out :-)

> you end up with very large dependency trees similar to JS. And we know how that went.

The problem in the JS ecosystem is not the number of dependencies, but the number of entities you have to trust. There is no real difference between having one big crate in a repo managed by one entity or fifteen small crates in a repo managed by one entity.

Sounds like the community still has to decide/write about what are best practises to make the compiler go fast?

What was your heuristic for deciding if a crate is too big or not?

Going by feel, same as when I have a giant function I decide to split up. IIRC the crates ended up between 1k-10k LOC

In OP the biggest crate after the split-up is 1,513 loc, with most of them a few hundred loc.

Reply on news.ycombinator.com