Vetting the Cargo | Hacker News Reader

Vetting the Cargo

lwn.net

73 pointstta38 comments

This sounds a lot like cargo-crev but without off-line cryptographic signatures, a significant downgrade in my view.

Edit: Yep: https://mozilla.github.io/cargo-vet/design-choice-faq.html#h...

None of the reasons given by Mozilla seem to justify the downgrade in security, especially since most can be worked around with crev which already employs secure and well-tested authentication schemes.

What also makes this situation peculiar to me is that it's being immediately rushed into Cargo proper instead of the usual way these tools are handled by the Cargo team (i.e. allowing multiple ideas to compete as third-party tools and maybe choosing one once a winner is clear). I understand the recent string of security issues might have played a role here, but I wouldn't expect their reaction to be steamrolling an inferior version of crev as a builtin tool.

I would really like to know what happened here.

Disclosure: I use neither tool, but I'm very interested in the security and health of the Rust ecosystem.

> What also makes this situation peculiar to me is that it's being immediately rushed into Cargo proper instead of the usual way these tools are handled by the Cargo team (i.e. allowing multiple ideas to compete as third-party tools and maybe choosing one once a winner is clear).

This tool isn't being pushed into Cargo proper, and I doubt it will be any time soon. Instead, it is a third-party crate called "cargo-vet". If an executable named "cargo-whatever" is in the search path, Cargo can invoke it as a subcommand "cargo whatever". See Cargo's docs for more info: https://doc.rust-lang.org/cargo/reference/external-tools.htm...

hopefully nobody adds a cargo-buidl package then!

Why would a user install a cargo-buidl package when cargo build already works?

Typosquatting is definitely an issue, but not in this specific case.

The binary can be named that without the package being named so, so it seems it could happen

If an intruder can rename files in your $PATH, the defense is already broken (e.g. they could do the same with any of .bashrc, .profile, anything in .local/bin and many, many other places). If a program can run seemingly innocent programs like less or sed, it already can run arbitrary commands.

Security story of developing on Linux is basically non-existent unless you do everything on a remote machine or at least in a container (and, oh irony: direct access from your user to the Docker daemon basically means having root for that user).

Well I was thinking that "cargo install foo" can already install cargo-buidl in your path by innocent means i.e. no Rust build script.

But with build scripts (build.rs) any crate indeed has arbitrary code execution at build time, so they can't be trusted, like you imply.

However, sandboxing might be coming for build scripts, and when that's done, the typo attack is still there (package "foo" installing cargo-buidl).

right, but cargo-buidl could be installed as part of a different package that also provides a more useful-sounding binary, no?

You need to explicitly install these tools, so you can't "accidentally" invoke cargo-buidl from a transitive dependency or anything.

It's not like a tool like npx which will allow you to invoke code in arbitrary packages.

yeah it's great that "cargo buidl" doesn't automatically try to download and install cargo-buidl but if cargo-buidl somehow ends up in your $PATH (perhaps installed as a side effect of another package or even some other package manager like pip) then it will run. Though I guess someone could also just as easily put a malicious cargo in your $PATH and that will work in most cases, depending on precedence...

Since the audits are designed to be used at a per project level and contributed directly into the VCS repo (allowing you to using git signing for example) I don't quite understand what additional off-line cryptographic signatures are required here (considering that Cargo's lockfiles already contain a hash of the crate which would prevent the project from getting an altered version of a crate accidentally and that SHA validation is being considered as part of vet as well https://github.com/mozilla/cargo-vet/issues/116).

That's fine. The biggest problem right now is getting people actually review their dependencies. If cargo-vet can make that easier by removing hassle of signatures and WoT, it's still a huge benefit to the community. It's possible to make cargo-vet <> cargo-crev export tools.

> Each new participant automatically contributes its audits back to the commons, making it progressively less work for everyone to secure their dependencies.

This is really exciting and I hope it gets adopted by all package ecosystems.

Of course audits can't guarantee to find the most underhanded "bugdoors", but it will still be a huge step forwards if third parties can vouch for various properties of the code you are about to install, such as it being reproducibly built from a tagged release on a public repository, with no Unicode homoglyphs or unexplained high-entropy strings in the code, and the unit tests all passing.

This will naturally lead to the question of who can be trusted to provide these audits, but such automatable checks could be done by almost anyone and their reputation could grow with time (which might lead to second-layer systems which track which auditors make the most accurate claims). Perhaps there will be companies that offer cyber-insurance against these specific threats, and use the premiums from that to fund the audit checks.

Alternatives to cargo-vet that has been mentioned before here on HN:

- https://github.com/crev-dev/crev

- https://github.com/vouch-dev/vouch

Anyone know of any more alternatives or similar tools already available?

I’m hesitant about this effort too, but for different reasons than the ones mentioned already. “Vetting” code for the kernel means something very different than vetting code for a normal application. I mean sure, you want to make sure that the code is functional and free of bad practices that may hide bugs or vulnerabilities, but that’s kind of where the mutual needs end. Kernel code in some cases may not allocate, or may have to avoid the use of certain vector registers. Locking and synchronization in the kernel usually looks pretty different from what you’d want to do in userspace. I’m not entirely against the idea of using third party crates in the kernel but it seems to me that you’d want something better than just “Mozilla ships this in Firefox and they got one of their security engineers to look at it so it’s probably good” that this vetting process seems to provide.

Having heard about this for the first time, I think it's a great initiative. However, I'm concerned about the liabilities of having vetted a crate only to accidentally (ie non-maliciously) miss some kind of vulnerability.

Could this lead to the auditor getting sued?

I wouldn't mind getting a legal take on cargo-vet and similar tools.

Does anyone have a link to Mozilla's audits.toml?

I found one in Mozilla's GitHub[1], but it only has five entries. Moreover all of the crates in the file were audited by their respective authors - which kind of goes against the whole idea of this thing.

Overall GitHub seems to only have six audits.toml files[1], two of which are the Mozilla one mentioned above, the other four are empty.

[1] https://github.com/mozilla/gecko-dev/blob/64f3b7d019700f4fe3...

[2] https://github.com/search?q=filename%3Aaudits.toml&type=code

The snide/sarcastic tone is surprising and unbecoming for what's normally a source of high-quality articles

Has there been any study of the effectiveness of code audits? Sensitivity and specificity in trying to find security problems, unintentional or intentional.

It's a question of whether you can have a process which can reliably detect if an arbitrary program will do X, and the answer is proved to be no: https://en.wikipedia.org/wiki/Halting_problem

In real life however we have to accept imperfections, in which good-enough is often better than not at all

With out mandatory user fields optional or signed audits I wasn't convinced how useful this was. I originally read the post viewing this as a way of sharing audits amongst the general public, in similar fashion to how the packages themselves are distributed. However, once I RTFM[1] (well some of it at least) I realized that is not the intended use case and that I was mistaken.

tl'dr

* internal management of audits.

* reading audits in from trusted third parties.

I'm glad that someone is starting to tackle this terrifying problem.

[1] https://mozilla.github.io/cargo-vet/importing-audits.html

If this does not have a way to track and filter based on who did the audit then it will wind up like the semantic web where anyone can tag a page with safeGoodQuality.

> A developer working on a function may suddenly discover the need to, say, left-pad a string with blanks. Rather than go though the pain of implementing this challenging functionality ...

The irony is palpable.

This feels like the equivalent of AdressSanitizer and similar tools for C. They fix a problem that shouldn't exist. At least not this extreme. C has the excuse of being old, Rust does not have that excuse. Using npm as an inspiration for cargo is just really sad.

> They fix a problem that shouldn't exist

Shouldn't exists because why? Because people should just be nice?

Supply-chain attacks are a real vector, one that seems to be able to have a larger impact by each day, as the OSS ecosystem grows and the dependency of using dependencies grow.

People want to be able to use 3rd party packages, like we've been doing for quite some time now. But there are a lot of them, and no (easy) way to manage exactly which one are "good" vs not, without manually going through each one of them, for each project.

Any ecosystem where you’re running code from unvetted third parties is susceptible to this problem.

We either need solutions to improve the supply chain safety or never use third party dependencies.

> Our dependency tree has steadily grown to almost four hundred third-party crates, and we have thus far lacked a mechanism to efficiently audit this code and ensure that we do so systematically. (-Firefox)

Wow. This makes me feel like I have to stop using Firefox.

I wonder if others feel the same, or have a different analysis. For example, is the situation with Chrome better?

It’s the current state of development as a whole. I assume most browsers and operating systems do this.

That is not the case.

In fact, Firefox itself did not do this until recent years.

It is true that modern JS development often works that way (as does Rust and some others), but it is not the norm in all ecosystems, and definitely wasn't in browsers.

I very much doubt that Firefox and most other internal software didn't do this until recent years though usually in a more ad-hoc manner, i.e. "oh look, that code does what I want ... but eh .. how can I add that as dependency, far too much work, let's copy it into our tree", which gives you all the downsides of the current state without any of the upsides.

I can tell you as a former Firefox developer that that was not the case. Yes, Firefox has some dependencies copied into the tree, but deciding to do it, and the process afterwards (including updating), were very careful and slow.

Of course there may have been exceptions I am not aware of, and developers are humans that can make mistakes, but that is the overall culture I experienced. It is fundamentally different to the JS/Rust/etc. ecosystem models.

> Wow. This makes me feel like I have to stop using Firefox.

I already had that urge in a powerful way after the last ESR update filled my browser with seemingly impossible to remove Bing and Google search hooks, among other obnoxious commercial b.s. I specifically use Firefox to not have shoved in my face at the browser internals level.

I'd probably be using Epiphany/GNOME Web full-time if it provided a noscript analog.

It's quite likely that this is an accurate desctiption of much development generally, and that Mozilla are simply more transparent about the situation than most vendors.

Though Google's deep pockets for security work do likely convey advantage.

If your standard is that third party code must be audited, you'll have quite a small selection of software to use. It's mostly all running on faith and reputation and counting on good intentions. (Like society generally!)

The specific issue described is a recent development. I'm a former Mozillian, and this is surprising to hear. In the pre-Rust days, this NPM-style fractal-of-dependencies approach wasn't a thing in Gecko/Firefox, and anyone naively insisting that this is just how things work in software development (and purportedly have to work) was someone who demonstrably didn't know what they were talking about. Looks like we've lost some of the "demonstrably" part.

I wasn't talking about Firefox specifically, I could have made that more explicit... But it's good to hear that Mozilla has good culture for this. If there is something you can say or link to about systematic third party code auditing at Mozilla, eg are results public, it would be interesting to hear. Or about how many vulnerabilities are discovered in code audits vs post-shipping security testing like fuzzing and other pentesting-y activities.

(Of course good control of versions is still a worthy goal for many situations even if you don't do this)

At times like these I'm reminded of the phrase “If you wish to make an apple pie from scratch, you must first invent the universe”

You have to rely on something in life, just like in an office building you can't realistically check the structure inside-out, or if you can, how can you make sure that the individual components are actually of the material they say they are? Have you double-checked your local water table yourself? Have you done geological studies to uncover vulnerabilities the subcontracting firm building the office may not have done correctly? What is the effect of local radio-interference or power quality on your equipment? Are your UPS-devices actually performing to spec?

> At times like these I'm reminded of the phrase “If you wish to make an apple pie from scratch, you must first invent the universe”

Sure, but it may also be a comparison between grandma's apple pie and the apple pie from the supermarket. If you never looked at the ingredient list of the latter, you're bound to be surprised at what you're getting. If grandma hand picked her apples from the back yard, you may be unhappy to learn about the pesticides used for commercially grown apples.

> You have to rely on something in life, just like in an office building you can't realistically check the structure inside-out, or if you can, how can you make sure that the individual components are actually of the material they say they are?

In that case, there are building codes as well as checks to ensure they are followed. While it is possible for someone to ignore those codes, there is also a cost for doing so if you are caught. For the most part, the software industry doesn't have building codes. If something fails, software licenses are generally written to avoid accountability. About the only constraint is the negative response of the market, but even that can be managed to some degree.

Reply on news.ycombinator.com