Rust module system encourages bad practices
dmitryfrank.com
dmitryfrank.com
Encouraging cyclic dependencies on the other hand “because it is easy” seems like a lazy cop out.
That having been said, I think my bigger issue with this is that this actually gets to be annoying after you do it, not just while doing it. Go manages external dependencies on the module level, which is a level higher than packages. For Rust, it's crates. So a repo with many crates becomes pretty cumbersome. (And a bit moreso if you're trying to maintain a Nix package for said repo, but that's partly the fault of Nix. Only partly, because I actually think Rust's toolchain makes the job of Nix rather difficult...)
As far as I can tell, all or most of these requirements are met by Cargo lockfiles as well, so what is the gap that makes Cargo/Rust difficult to do well in Nix?
The other thing that's weird is that you have to embed the lockfile into the Nix derivative, at least in some cases. I actually can't remember why that has to be done, but presumably there is a good reason.
I think most multi-crate projects are only sorta accidentally including multiple versions of the same crate, which is why it's rather unfortunate that there isn't some abstraction between crate and module.
However, the Cargo team has been stretched a bit thin; back in April they added two new team members, and so things look like they're going in the right direction again: https://blog.rust-lang.org/inside-rust/2023/04/06/cargo-new-...
I've got other gripes with cargo.
- A go mod download equivalent would be nice so that we can actually reliably cache dependencies in ci. - More sane linking tooling when statically compiling, eg. Openssl. - Build.rs magic at compile time. - No good way to publish a workspace of crates without actually having to publish everything. - General workspace versioning unfriendliness. It is tricky to bump versions correctly in a workspace.
Probably some more. Most of these already have upstream issues but most have been stale for a while
A go mod download equivalent would be nice so that we can actually reliably
cache dependencies in ci.
Basically you don't want a source change to invalidate your dependency fetching, which can occur if you use cargo fetch and just caching the target, especially with docker, as it doesn't have granular enough caching mechanisms.
Caching ~/.cargo is great tho =D
Its not.
with Go you have ONE go.mod file, at the module root. you may have 100 sub packages or sub sub sub packages, but still only ONE go.mod file. then any time you need to add a new import at ANY level, you just go "go mod tidy", and DONE. please dont try to compares apple and orange and say they are both apples.
Any of these subpackages, if they don't have a go.mod file, would require defining their dependencies top level as well, and when you go get the package you would need to pull everything, even if you only need a subpackage. Which usually isn't a problem, because golang is super fast to compile.
I get that they aren't identical, but tbh they are pretty close
What does go mod download do? cargo fetch/cargo vendor don't do the trick?
This is the upstream issue in question: https://github.com/rust-lang/cargo/issues/2644
Cargo fetch and/or cargo-chef kind of does these things, but not in a super intuitive and easy to use way.
I can't for example shove all of the definitions into single file then use it in other files without using sub-modules for no good reason, I much prefer "module = directory" approach to "module = file"
I much prefer "module = directory" approach to "module = file"
First, we allow sharing `Cargo.toml` fields across the workspace since 1.64.0 [0]. In the upcoming 1.71.0, we extend this so that `cargo new` will automatically inherit fields when creating new `Cargo.toml` files [1].
From a different angle, we are also adding support for single-file packages though this is more meant for binaries [2]
[0] https://doc.rust-lang.org/cargo/reference/workspaces.html#th...
For instance, if you make more crates and they all use the same dependency, you have to manage it all in the multiple Cargo.toml files.
But overall, they're just two different systems with two different tradeoffs. I prefer writing Rust and I prefer the way I specify deps in Rust, but I prefer the ergonomics of packages in Go.
But is it so simple that it can be done often without having to go back to the docs and look up something, then making sure everything in the crate makes the build system happy? We get better at doing things the more often we do them, of course, so there's some inflection point where "often enough to remember how to do it" and "simple enough to be able to recall" cross.
With Go, making a new module is just creating a new directory, something programmers have done countless times, and requires no cooperation from the build tool. The only requirement is that files in directory have the same package declaration.
I read it more as the maintenance overhead when publishing. You need to bump version, check dependency versions etc.
I'm speaking from personal experience here. The more I split my crates into sub-crates, the longer the release/publish process takes.
There are some parts you can automate with e.g. cargo-release but I now think a lot harder before cutting a release to crates.io.
I also have the gut feeling this is happening throughout the ecosystem.
I have a lot more overlays in my Cargo.tomls now than I used to, two years ago.
I.e. there are PRs and fixes in the repo. that you need but the author hasn't cut a release.
This may be coincidence ofc but I can't help but think there may be a connection.
I'm a bit confused by this as I'm used to seeing the opposite complaint, that Rust packages are too small. Maybe if there were examples it would help.
And while I do think we should explore ways to allow lower effort packages ("cargo script" is one component of this [0]), I see allowing submodules as a feature though maybe thats my Python/C++ background showing through. I do like that it allows both low overhead modules in your API (just a file) or allows you to separate a module out into multiple files for clearer organization (where cycles become important). While I don't know if there is a feasible design for this, but I do wish we could specify what modules are cyclic vs acyclic.
By "packages" here you're probably referring to the public crates on crates.io, right?
Indeed, those public crates are not big in general, but that's not the point of the article. The article is rather talking about large, multi-crate projects, organized as a Cargo workspace. I agree it could have been clearer in the article.
As an example of good practice, Go is pointed to, which is an ecosystem full of people splitting lots of stuff into separate libraries that make no sense to the end user as a separate library and in practice serve the role of submodules. This I would call bad practice, if not for the fact that Go disallows you from doing it any other way.
If Go had a central package index instead of just using GitHub, it would be a very confusing place, full of 'private submodules'.
Been years since I’ve done much Rust. Does this mean bumping dependency versions thus becomes a larger chore in a project?
Contrast this with C++ where the DAG relationship is enforced at much lower levels (e.g. Bazel cc_library, CMake target, which in normal cases correspond to a cc/h pair), shifting the friction (of having to deal with deps) much earlier in the dev cycle, which then discourages you from (incrementally) splitting a file at all. I've seen too many times this resulting in 5000+ LoC cc file.
Naturally you can only enjoy them today on VC++, but the rest will follow.
Personally, this strikes me as bizarre. It is only an "illusion" if one expects a module tree structure to map one-to-one with dependencies. No one should expect this, as I will explain with three uncontroversial and widely known points:
A. Are there any languages where parent-child relationships between modules necessarily imply dependency semantics? I can't think of any, nor can I envision any sane language doing so. [Note 1]
B. In Rust, like with so many languages, when a module is nested in another module, that means the inner module is scoped inside the outer module. It is about namespacing, not dependencies.
C. Dependencies are created when the contents of something in a module refer to another module.
[Note 1]: How would such a language specify more than one dependency? A node in a tree can only have one parent, after all. This would rule out a one-to-one mapping.
(1) https://github.com/rust-lang/rfcs/issues/493 (although the comment section is kind of unhinged and goes off on distant tangents to modular typeclasses and first class modules)
(2) http://smallcultfollowing.com/babysteps/blog/2015/01/14/litt...
On a tangent I recall struggling with cargo features in the beginning. I found them to be hidden and mysterious. Even searching for “rust features” sucks.
Took me a bit too at first.
Interesting since I think Rust very much encourages the opposite, which is why so many crates like hyper, regex, and others, are actually composed of many crates.
I don't really get the main thrust of the article. Go forces directories to map to packages. That sounds... awful. I guess it "encourages small packages" because you can't even organize them into folders? IDK, I feel like I'm missing something - particularly, I'm missing why big or small is good or bad, or why creating a crate is onerous.
I wouldn't pick Go as an example of how to build a module system. Far from it. ML languages yes.
Anyways, adding a single line command per compile unit package is not a "cost."
Edit: P.S. Also the module logical system in Rust is way much more expressive.
Also, in this case, Go doesn't exactly protect the developer from anything. One could create a single package with tons of files in them, and it'll keep working. But the way Go package system is designed, it just subtly suggests to the developer that it's not the right thing to do: having a package (i.e. directory) with 200+ files in it just feels wrong by default. One trait of a well-designed system is that it's difficult to abuse.
The other stuff here is mostly just a matter of convention. Every large Rust codebase I'm familiar with absolutely does avoid creating huge crates, so I think the author is just mistaken about what constitutes standard Rust practice.
$ echo -e 'package main\n\nimport _ "golang.org/x/image/draw"' > main.go
$ go run main.go
main.go:3:8: no required module provides package golang.org/x/image/draw: go.mod file not found in current directory or any parent directory; see 'go help modules'
$ go build main.go
<same error>
With rust, you can also technically just call 'rustc' and never use 'cargo', but just like it's a pain to do that in go, it's a pain to do that in rust.
A bit esoteric, but I like that option, especially when working on an application and its dependencies in parallel at times.
1. go mod init something # create go.mod
2. go mod tidy # create go.sum
3. have fun
[0] this is probably a bad opinion from someone who's honestly never designed large systems, but it just seems to me that sometimes cycles are inevitable, you just have to be careful when they arise and minimize the times they happen.
2) what if 2 modules in go share a common dependency? It's not clear to me how that's handled in a file system hierarchy. But thats beside the point anyway.
All this rust to learn Rust seems more like a gold rush to me.
Go seems to be the python equivalent of systems programming
The real problem is that you can't use crates to improve incrementalism when you have a type in one logical part of the module hierarchy implement a trait from another logical part. Due to orphan rules and coherence, it is not possible to implement a foreign trait for a foreign type.
And that said, just because the compilation unit is a crate doesn't mean it can't be compiled incrementally or with parallelism. That's a separate problem.
Well you can implement a trait for a type from another logical part either in the crate of the trait or the crate of the type.
What is the use-case for implementing a foreign trait for a foreign type if you have both under control?
I wanted to separate the type, the trait and the implementation of the trait in different crates. I was forced to use newtypes but newtypes need a lot of boiler plate. It was painful.
In my opinion this is a larger barrier than having separate Cargo.toml files.
I usually put the traits in a separate crate, but keep concrete types and implementations for them in the same crate.
What is the use-case for splitting a type and it's implementation into separate crates? (except for cases where the trait or the type are out of your control, then one indeed needs to use the new-type escape hatch)
The problem was that the trait implementations of the types was something like not a core task of the types themselves.
All this rust to learn Rust seems more like a gold rush to me.
Go seems to be the python equivalent of systems programming
Also note that modules are not per se compile units.
Cf. See https://doc.rust-lang.org/rust-by-example/mod/split.html
Edit: typo.
Move your shared dependencies into another crate.
Not to mention the whole bonkers idea of having package names bound to Github repos and source only.
Again not claiming superiority of one tool over another... But the user experience for rust has been a lot better for project organization for me then most languages. My biggest gripe is, the rust docs somehow make this incredibly difficult language easy to learn, but the sections around code organization are really really hard for beginners or anyone who hasn't sunk time into it and completely omit the 10 meter view of how to organize rust code.