I believe running mach bootstrap again will just work (if not we can make that happen).
I don't think it's a big deal that the possibly-sudo-requiring step is kept separate and not automatically invoked on build.
I don't think it's a big deal that the possibly-sudo-requiring step is kept separate and not automatically invoked on build.
You get reproducible builds because the build happens in its own sandbox, ideally. But it's not absolute.
There's a continuum here: it's not awesome to have builds from source for every library if you know you're building inside a defined Docker image, for example.
So yeah, would be nice to have, but I'd be wary about making it the default.
I find it fascinating that lessons aren't being learned from build systems which are built in the enterprise like Bazel, Pants, etc (honestly, I'm disappointed one of those two wasn't just adopted by the Rust community - unclear but I think they didn't know about them?).
Similarly, notice how "Java shops" have for many years enforced ever increasingly complex build pipelines (Javadoc, Junit, JaCoCo, Findbugs, ...). But these tools are slow, end up with long standing bugs when versions bump, and generally end up limited.
I find it fascinating however, that only two of the above steps (Docs, Tests) are first class in Rust. Why only those two? Why not learn from the whole Java build pipeline? Why does the compiler's test infra not spit out metadata about lines/branches covered? Why is clippy, like Findbugs, separate from the compiler? Isn't "better code" by definition better for everyone.
It should be relatively easy for these to be built into the compiler. While some like test coverage data is really hard to get as separate tool (especially when optimizations are on). However, Rust developers have valid points like "separation of concerns". Its hard to argue with them, after all we are only on the sidelines, they are actually in the thick of it. But again, there just seems to be a divide, and its interesting to see which features/lessons are adopted, vs which features/lessons are "not worth the trouble".
FWIW (my stab at a reason for this divide), I think its largely a problem of peers. In the enterprise, you work with great developers and you work with not so great developers. Most of these are fresh college/code-bootcamp developers but there are some rare bad experienced developers. At least in the field of building languages (maybe open source in general) you get to be very selective. Essentially, you only get peers who are experts (even if some of them do start off making changes while in college or without any formal education - also keep in mind the selection bias of people curious and motivated to help a project they aren't being paid for). Compounded with few strict deadlines, this basically means everyone is on their A-game all the time. eg. "Why would you need metrics about how many lines+branches of code are tested? and you want to potentially fail the build if it doesn't meet some threshold?? Some things are hard to test! Shouldn't the implementor get to decide how much testing is the appropriate amount of testing?"
P.S. Please, anyone, correct me where I'm wrong. I too am on only one side of this divide.
So instead of choosing a large feature set at the beginning and the box it through, they choose to have a more thin compiler, standard library etc. but make it extendable, so that it can be extended and integrate with other systems. Also you should not forget that Rust is still very young for a programming language so cargo is everything but done. Cargo is more a unified interface combining the compiler and the crates package repository than a full fledged build system. Given the available resources and time for building rust, I thing choosing to (for now) not include a fist class rust coverage system, debugger etc. is a sane choice, especially because all this things are available as external tools (debugger=gdb, coverage=kcov, etc.).
Also there are discussions/issues about enabling the integration with "enterprise" build systems, they just currently don't have a high priority and happen mostly in the background. So it's probably just a matter of time until you can use rust with a "enterprise" build system like Bazel.
Wrt. clippy integration in rust, it is mostly about keeping the compiler smaller and easier to maintain. Through it should be noted, that all "important" lints are in rustc, clippy just provides more useful ones. By keeping both projects separated it is possible to iterate both of them independent of the other. Which can be quite useful in open source development. Also don't enterprise systems also have a separate linter? (And no "better code" is not better code for every one, as the definition for what good code is can change quite a bit depending on the person, e.g. wrt. variable shadowing)
Lastly I don't think your reasoning about peers holds, you don't really have the choice to be selective about people working on the project, at last not if you have open source project on the scale of rust. Through you might be able to have more strict requirements for code quality. Nevertheless given how rustc catches a lot of possible bugs, and how to community is, I would be surprised if there would not be some less experienced programmers contributing to the compiler. But then at last Mozilla also uses rust in production, so some parts have deadlines and even if not it's quite different for them then a `A-game`classification. Also it's not so, that you can't have metrics about lines/branch coverage, failed the build on certain thresholds, etc. It's just not necessary a build-in feature, through available nevertheless. Also it would be really strange if a programming language / build tool would decide which amount of coverage is ok, it's something the project manager/programmer has to decide (once) and then configure the build tool to enforce it (I most likely did misunderstood you on the last line ;-) ).
Ups, I wrote much to much.
TL;TR: 1) rust has less (programmer) resources 2) it's still young, tools like cargo, or the testing API are everything but complete. Through they do there job good enough for now.
This is expressed pretty nicely, thank you. I was struggling to express this in my other comment. rustc has all the lints which:
- Everyone mostly agrees upon
- Have rare false positives
- Are of the kind you'd expect from a good compiler
It doesn't have 150 lints because 150 lints would be annoying, and would drown out the "important" lints. I work on clippy, and I find this distinction very useful. When rustc tells you to fix something, you fix it. Clippy lints help /inform/, and you may ignore them often. But you don't always need to listen to clippy. If I wrote a bot that fixed all rustc warnings I'm sure people would be fine with just blindly running it. I cannot say the same about clippy (not because of bugs), and that's a good thing.
The plan instead is to polish clippy and distribute it via rustup.
Yes, you can add new lints with an RfC. However, the bar for inclusion is very high.
I'm totally okay with the status quo, but I very much disagree that there is scope for uplifting lints to rustc from clippy with the current policies.
On one hand, I see how it is annoying when a compiler complains about code patterns that are not worth errors and could be false positives. OTOH, I feel like a language designed for safety should try to warn about as many bugs as possible. Would any of clippy's lints pass every Rust crate tested on Crater? Maybe those could be come rustc errors.
For fun:
Lint, a C Program Checker (1977)
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.56.1841
lolnope
Well, there are lint-checking-lints, and there are very few linters out there, and clippy passes clippy, so those would pass. A couple other obscure lints that check for things that in practice never happen. Too obscure to be included in rustc I guess.
IMO a lint passing all of crater is a very good reason for it to not be included in rustc.
What specifically makes you think lessons aren't being learned?
> honestly, I'm disappointed one of those two wasn't just adopted by the Rust community - unclear but I think they didn't know about them?
Bazel had its first release in March 2015, and wasn't even considered beta ready until later that year. Pants had its 0.0.17 release, the very first public one, in July of 2014.
Cargo was announced in March 2014, earlier than both.
> Why only those two?
Well, those two are necessary. Other tooling is nice, but not the bare minimum needed. We need to stay focused, but more on that later.
> Why not learn from the whole Java build pipeline?
I have not dealt with the Java-specific tools you're talking about in a very long time, but pipelines have advantages too. A few slow tools doesn't mean the whole concept is bad.
> Why is clippy, like Findbugs, separate from the compiler?
Rust cares a _lot_ about stability. Adding new lints is extremely difficult, because they can break people's builds. Clippy being a separate tool means that they can iterate on lints at a different release cadence than the compiler itself, and aren't bound by our mega strict breaking changes policies.
> In the enterprise, you work with great developers and you work with not so great developers.
I assure you, it is very much the same in open source. ;)
So, we do very much care about Rust's usage in enterprise scenarios. For example, Mozilla in many ways feels exactly like an enterprise customer of ours. It was a huge amount of technical (and social) work to get to this point, and a lot of that involved negotiating how the builds would actually work. Our perspective is that Cargo is a world-class tool for building code, but that's only possible because it's focused on _Rust code_. So the strategy should be, how can we best integrate Cargo with these more generic build systems? We've already taken a number of steps forward on this front, and there's more to come.
Most of these features lacking in Cargo are due to a lack of some kind of champion to lay out the specific needs; we don't want to just build features and hope they're useful. In the past, we've had several people come to us with "here's my issue getting Rust in my workplace", and that experience has been valuable. We're a small crew, we need to focus in order to keep shipping quality stuff. We don't have the ability to just toss some people on it, for whatever value of it, and get it done.
> Cargo was announced in March 2014, earlier than both.
Thats fair, and a part of what I meant about "they didn't know about them", because working for Amazon right out of college, I've been using things like Bazel/Docker/Travis for 5+ years now.
I actually meant it as a discredit to people who work in the industry for not exposing these tools earlier. I did a poor job of communicating that (it was after all like 3 am :P).
Furthermore, Bazel and Cargo are aimed at slightly different use-cases. Cargo is tailored towards Rust, is integrated with a Rust package repository, and can use it's knowledge of Rust to allow for builds of pure-Rust projects with a lot less configuration. Bazel is a more general purpose build system, but that means it requires a little more configuration. There's no reason you couldn't use Bazel for building a larger project that includes Rust and C, C++, Java, or other languages that need to be built, but Cargo makes it a lot easier and simpler to handle pure-Rust projects than it would be in Bazel.
Many of the other tools that you mention don't require just more support from the build tool, but also significant engineering effort on the part of the compiler, or a separate static analyzer, or other development process tools. These tools are desired, there just isn't anyone who has had the time to design and build them.
As to why Clippy is separate from the compiler, there are a couple of reasons. Adding more lints to the compiler, especially ones that are turned on by default, can cause problems as well. Any time you add a new lint, you may break people's code that has #![deny(warnings)] turned on. Even if people don't, a lot of people will still try to rewrite code to make it comply with lints; but that in itself can cause problems. I've seen many bugs introduced over the years by people trying to do too many lint cleanups at once, and making mistakes in some of their cleanups. For this reason, the Rust compiler is fairly conservative about adding new lints..
That said, it sounds like there are plans to integrate Clippy with the compiler and cargo, but probably as an opt-in separate command rather than being included in the default set of warnings: https://www.reddit.com/r/rust/comments/5ibr2a/cargo_check_ha...
Code coverage is another example. You can use some existing code-coverage tools with Rust (https://users.rust-lang.org/t/tutorial-how-to-collect-test-c...). Because there are tools out there that work, there's less of a pressing need to get code coverage integrated with the Rust toolchain; I think that it would be a good idea to do eventually, as it will make it easier to do cross-platform and out of the box, but for now it's not the highest priority.
So, I don't think that any of these omissions are due to a difference of philosophy; just a difference of focus. Rust is all about admitting that programmers aren't perfect, and that better tooling can help avoid mistakes and thus make developers more productive. Right now, that focus is on things like improving the compiler (adding MIR, which allows for fixing several bugs in the compiler and doing some Rust-specific optimizations before getting to LLVM), implementing the Rust Language Server which can be used as a backend for IDEs, improving the ability of the compiler to do incremental and parallel builds, and adding language features with a focus on making Rust easier to use.
They didn't exist at the time. cargo was built using lessons learned from other package managers, however.
> Why is clippy, like Findbugs, separate from the compiler?
The compiler didn't want to bloat its lints. Being out of tree let us iterate quickly. It lets us work on extremely useful lints which as a side effect have false positives (there are a lot of patterns it catches which might be buggy but sometimes have legit use cases that are hard for clippy to detect and ignore). It lets us add controversial lints. In general a separation between the "core" lints (which everyone usually agrees on passing, and cover a small set of bare-minimum issues) and the clippy lints (which not everyone agrees on, and cover a wide range of issues which don't always apply to your codebase) is nice to have.
There are many reasons to not want to run clippy, one of them being that's it's too strict/annoying. In Servo we can't run clippy because someone needs to go through it and update Servo over the thousands of warnings clippy produces when run on Servo (many of which are false positives). I occasionally do some of this, but clippy grows pretty quickly too so each time I do it there are new things. (I'm waiting for rustfix to mature so that I can automate a lot of this).
So, to me, clippy should be something you have to decide to use.
We do plan to make clippy part of the rust distribution (bundled with cargo and friends), it's just not happened yet since it's blocked on a bunch of things.
Clippy not being part of the default pipeline is just an artifact of its immaturity. It's still used by a lot of people despite not getting free publicity from being a default which is promising, though.
I don't see why there's much value being assigned to "first class" tooling in Rust. code coverage and clippy are a `cargo install` away.
> Most of these are fresh college/code-bootcamp developers but there are some rare bad experienced developers
There are a LOT of new/inexperienced programmers in the Rust community. You're right that there's still some selection bias, but Rust tends to be pretty welcoming. I've mentored people who've only done very basic programming in making Servo pull requests. Some of the sporadic clippy contributors are pretty new to programming.
> Why would you need metrics about how many lines+branches of code are tested?
I have never seen opposition to code coverage in Rust in that form. In general folks in Rust are happy to have more kinds of checks lying around.
It's not part of the pipeline mostly because it's a cargo install away. If your project needs it, it's pretty easy to make it part of your workflow.
There is an argument to be made that making it part of the pipeline will mean that more people will be driven to use it, which is great. You'd have to take it up with the tools team and community if you think this is something that needs to happen; I am mostly ambivalent about the idea.
I was trying to discuss the divide (maybe come up with reasons for it - potentially improve it), not the lack or need for tools. I was just providing concrete examples using Java vs Rust build pipelines. It might have been better if I had stuck to abstract examples like: "I find it fascinating when something like X is so obviously a priority to me(us), but it takes a lot of effort to convince others that it is a priority at all."
I hope that is more clear.