Hacker News Reader

Comment by 013a | Hacker News Reader

If there was a sports-like betting site for future decisions that major open source projects are going to make, I would put a thousand dollars on "Cargo/Crates will implement namespaces."

The developers have convinced themselves that they're right on this, having, I suppose, not learned from the hundreds of package managers that have come before them. They're wrong; full stop, no qualifications, no asterisk, no footnote, they are totally wrong on this and it is inevitable that their opinion will change.

But, this is the nature of open source (and even startups). New projects start under the (oftentimes correct) belief that they can do something better, which experiences linear extrapolation into believing everything you do is better through virtue of being unlike what everyone else is doing. We see the end-state of Maven, NPM, etc; we don't see the war that those projects went through to reach this state. History is elided, forgotten, then repeated.

View on HN

> The developers have convinced themselves that they're right on this, having, I suppose, not learned from the hundreds of package managers that have come before them. They're wrong; full stop, no qualifications, no asterisk, no footnote, they are totally wrong on this and it is inevitable that their opinion will change.

Comments like this are why this discussion has devolved over the years. It's "all heat and no light." It does not help convince anyone that you're right, and if you want this to change, I would suggest that you change your approach.

The team has stated that they are open to implementing namespaces, but that the people who advocate for them need to address the concerns. This hasn't been done successfully yet, and is part of what this post is trying to help out with, by cataloging some of the large amounts of previous discussion.

What concerns? Most people don't care because they feel that the ship has sailed, the mistake is set in the concrete. Transitioning thus becomes the biggest conceivable concern for me.

What are the other possible concerns? The only comments I've seen on this is "It's not any different, people will just squat namespaces" to which I reply: reserve namespaces up front for Rust nursery and bespoke crates.

That has always been the biggest benefit of namespaces, when i type "std.rand" I never have to worry about an attacker squatting "std.rnd". But I could just mistype the namespace! Fine, I could, but all the obvious mistypings could _also_ be reserved!

The article has some discussion on all of the various bits of the problems in this space, and has links to previous discussions.

It seems from your comment (and I may be wrong!) that you're purely talking about the "use namespace to solve squatting" issue, which is only one dimension here. Some people want namespaces and don't care about squatting. Some people want to solve squatting through other means. Some people do see namespaces as a solution to squatting, while others disagree.

I think the point of the post is that whatever the reason, every other package manager that has started out without namespaces has eventually implemented them.

The reasons may be unclear, but it seems likely that eventually the crates.io team will receive the same reasons that the other package managers got and implement namespaces.

That’s just not true though. Rubygems does not, and neither does CPAN, I believe. It doesn't look like PyPi does.

There's been some discussion around it for PyPI, but it doesn't seem like there's a consensus that it is necessary, or even on what the set of problems that should be addressed by it is.

... sounds familiar...

> The developers have convinced themselves that they're right on this

This is a complete misrepresentation of the issue. Adding namespaces to crates.io is a social problem, not a technical one, because adding them means you now must do the work of arbitrating disputes over an identity layer, which is a job unfit for part-time volunteers, which is all that crates.io has. The only alternative to arbitrating your own identity layer is to tie yourself to some other preexisting identity layer (e.g. GitHub, DNS), which raises questions about the notion of project ownership, the potential transfer of ownership, the immutability of the package registry, and all the while just pushes the "race to register" to a different sphere.

If crates.io had full-time employees rather than only volunteers, I'd be all for paying someone to handle identity problems in exchange for the existence of namespacing. But who's going to pay for it?

I agree with the main thrust of your post, but there is some level of technical problem because the Rust language doesn't support namespaces in crate names themeselves, and so you have to deal with that (or not...) in some capacity too.

Obviously we're all talking hypotheticals here, but Rust-the-language doesn't need to support crate-level namespaces (which isn't to say that some first-class support couldn't be imagined, but again: hypotheticals). Rust links to libraries (via explicit or implicit `extern crate`) that it finds on the system via the ordinary OS-level library search path mechanism; it's up to the package manager to put those libraries on the system in a place (and with a name!) where Rust can find them. As long as the package manager can turn the registry namespaces into something usable from Rust, then there's no need to add anything to the language itself. A package manager could decide to put a namespaced package "foo/bar" on the system as merely "bar", and require the user to manually disambiguate when colliding (using ordinary rename rules, such as is already supported by Cargo). Alternatively, a registry could forbid uploading packages whose names internally contain underscores, and then the package manager could install "foo/bar" as a crate "foo_bar", which has no risk of collision (well, with packages from that registry).

This is why the conflation in the title of the OP here is especially annoying; package management is a concern that the language doesn't necessarily benefit from being aware of, hence the separation of the compiler and the package manager. :)

> As long as the package manager can turn the registry namespaces into something usable from Rust,

All I am trying to say is that this is a technical problem, in addition to the larger problems, which you correctly identified are social.

> Rust links to libraries (via explicit or implicit `extern crate`) that it finds on the system via the ordinary OS-level library search path mechanism

Nit: rustc gets passed each crate available to be used or extern crate via an --extern flag that specifies the name of the crate as well as the full path. The path is optional and if not specified rustc performs a search, but the name is mandatory. Cargo only uses the mode with both path and name specified. This also isn't an edition 2018 thing, it exists on older cargo versions too. It's possible to specify the same file under multiple names, then it's available to Rust programs under multiple names. It's also possible to specify multiple paths for the same name, but then only one will be taken, the first one which is available.

Also, note that Cargo works in terms of packages, and rustc in terms of crates. A package can contain up to one library crate, and the names of the two don't have to match.

I'm also under the impression that the crates.io team was somewhat reluctant to implement this because it was viewed as a potential security risk from a social engineering point-of-view, even if there were people dedicated to it full time.

That said, I'm having a hard time finding the Rust Internals discussion thread, so I can't tell if I'm misremembering somehow.

Edit: Actually, I think this was in context of re-assigning crate names that may have been "squatted", and how making decisions around what amounted to team member's arbitrary decisions around re-assignment could cause issues. Scratch this, please.

> The only alternative to arbitrating your own identity layer is to tie yourself to some other preexisting identity layer (e.g. GitHub, DNS)

Devolving identity down to DNS has been a really good decision for Go. I'm not sure why everyone doesn't do it. It's like XML namespaces, but actually useful.

Organizations live shorter than packages, just look at all the com.sun stuff in Java. And even in the absence of corporate changes, you may well want to transfer a package to a new home.

> Devolving identity down to DNS has been a really good decision for Go

If it works for go, the because go is not old enough. Domain registrations lapse quickly and then someone else can buy them. That's very close to the worst that can happen for trust in packages.

Ok, then how about Maven Central? They give you publishing rights to a particular group ID by verifying your ownership of the corresponding DNS names. They've been around long enough, I think, even if Go hasn't.

Maven Central uses that as a first-time verification (actually, it's OSSRH that requires this, technically there are other ways to publish to Maven Central).

Golang's package system depends on the current DNS records.

With maven, there are plenty of old projects that do not own the domain names for their group ID, and that doesn't cause any problems - they are well-known, established projects, and there's no reason to introduce breaking changes to everyone just because they want to go through some DNS naming changes.

Because of this, there's little reason to fear that if you publish your project today as "io.github.my-project:my-module" you may find yourself having to break all your users if github blocks your account tomorrow (or just changes their URL schema) and you move to gitlab.

For better or worse, nearly everyone uses github, which also provides second level name spacing.

If you lose your domain, spreading bad software is the least of your concerns. They can reset all email passwords and access every service you’ve ever used.

> If you lose your domain, spreading bad software is the least of your concerns.

But if you lose your domain spreading bad software is the top of my concerns.

Either the package manager site allows email resets and the same issue applies, or it doesn’t and you can be locked out forever by forgetting a password. Both failure modes suck.

You're still assuming the domain owner's participation.

How many domains lapse while the owner is active and responsive in the community?

You just agreed with my original point that outsourcing identity to DNS/TLS is a good practice!

> If you lose your domain, spreading bad software is the least of your concerns.

OTOH, it's probably higher on the list of downstream code users’ lost of concerns.

> They can reset all email passwords and access every service you’ve ever used

Not if you dealt with all that before letting the domain lapse. And if you don't, more to the point, that's your problem, but not the package manager user communities problem. And that it is a bigger concern of yours than the impacts on the package manager user community is exactly why that community might want to protect themselves from you neglecting their concerns..

Not really. Look at the protobuf 'renaming' and think about how silly that is.

Protobuf renaming?

I was referring to this:

https://blog.golang.org/protobuf-apiv2

> The github.com/golang/protobuf module is APIv1.

> The google.golang.org/protobuf module is APIv2. We have taken advantage of the need to change the import path to switch to one that is not tied to a specific hosting provider.

Basically, Google realized that it's not a good idea to tie your package's identity to GitHub's URL scheme. This is easy for Google to fix, but the majority of Go packages suffer from the same problem, and it is much harder for a small project to start maintaining a DNS record that they control to point that to GitHub.

Not to mention, if GitHub decided to change its URL scheme for some reason, almost all Go builds on the planet would have to change.

And of course, the whole problem stems from relying on DNS to name your packages. No one wants to bother with DNS, so they go for the low effort option and let some hosting provider handle it for them, without caring what happens in 10 years time.

Are namespaces somehow more relevant to police than package names?

I don’t see how namespaces would be different?

> They're wrong; full stop, no qualifications, no asterisk, no footnote, they are totally wrong on this and it is inevitable that their opinion will change.

This sentence takes the place of a reason. You could have offered a reason why you were right or a rational argument about the matter. Instead, you basically yelled "I'm right, they're wrong! I'm right, they're wrong!".

The GP made nothing more than a prediction about the future.

Let's let the future determine whether he was right or wrong, no need for squabbling.

No, their first paragraph was a prediction. Their second was a assertion of fact without any evidence provided. They could have hedged that statement with "I think", or "in my opinion", but instead they went with "They're wrong; full stop, no qualifications, no asterisk, no footnote, they are totally wrong on this". Strong assertions require strong evidence, and not providing that evidence deserves to be called out. At least that's generally the consensus of the community here, as I see it.

Every package manager that came before cargo/crates.io is not enough evidence for their assertion?

Not if they don't actually mention it, no.

It's not that evidence doesn't exist (it may or may not, and people may or may not agree that something is evidence), it's that none was provided. Without that, all you get is people yelling that they're right and someone else is wrong. Evidence lets people move the argument to useful territory.

> Every package manager that came before cargo/crates.io is not enough evidence for their assertion?

Since not every package manager that came before it is consistent with the assertion, no.

I am not arguing that this should happen; I am asserting that this will happen. The reasons why it should or shouldn't happen don't matter when it will happen.

If you'd like to: Go back and re-read my comment, but re-read it in the mindframe that I'm against the change. It still holds. I didn't provide reasons why they should make the change, because I don't have a stance on whether they should or shouldn't. I am not qualified to say whether its the "Right" decision; just that Rust is on the wrong side of history with this one.

You have to admit the possibility that people will give up on rust before it happens (possibly because of this issue).

Just because they will do it eventually doesn't mean they are wrong now. It's entirely possible that a flat namespace is reasonable decision for a small registry with few resources, but a bad decision for a well funded registry of an ecosystem that had a few more years to develop.

> History is elided, forgotten, then repeated.

This seems like human nature. We continue to do the same thing with client side / Server side rendering. When it comes to computing, we oscillate a lot. In all these cases there is the desired system (lack of namespaces in a way they doesn't absolutely kill you later), we pursue it, we inevitably fail, get burned, revert, forget, repeat.

In the case of namespaces, this is a tale told a thousand times. To the point where I think people don't even want to try to fight for them. What's the point? They shall forge forward, namespace free, until it hurts enough decision makers. Then we get namespaces.

Now, back to Ruby for me. Where anything anywhere can just yolo reach into your namespace and mess your whole life up.

They're wrong; full stop, no qualifications, no asterisk, no footnote, they are totally wrong on this and it is inevitable that their opinion will change.

To be honest, that's an impressive oratory monstrosity you've produced there. No argument, no nothing, and nevertheless it sounds completely brain-shutting-down conclusive.

I don't think the rust people are deserving of it. They've done impressive work with minimal resources, and they have shown the ability to create consensus that's better than any of the individual parts that contribute to it. So I'm going to trust them in this for now, and maybe, they'll evolve in the future.

Nevertheless, I have to admit respect for that sentence, in the same sense I'll admit respect for an earth-shattering nuclear explosion. You might want to run for US president if you produce more of them, though I'm not sure if that's praise or damnation.

Respect for a sentence I would expect from my 5 year old?

One of these years I'll understand this place! I just know it!

A thousand times this.

Why are so many things in software three-steps-forward-two-steps-back?

> If there was a sports-like betting site for future decisions that major open source projects are going to make, I would put a thousand dollars on "Cargo/Crates will implement namespaces."

There is, sort of! Assuming you choose a semi-reasonable timeframe (5? 10 years?), I’d be happy to take the other side on Longbets.

I believe there’s even a reasonable argument to be made that your bet is societally important, as the rules [1] require, based on your final paragraph. Roughly your take is that over time, efforts realize that the naïveté of youth misread the battle scars of what came before. crates.io having namespaces or not is just the mechanism for this debate :)

[1] https://longbets.org/rules/

Reply on news.ycombinator.com