Five years with Rust
words.steveklabnik.com
words.steveklabnik.com
If you don't read anything else programming related next year, read the Rust book. Even if you don't need Rust, there's a lot you can learn from it.
It's good to hear that people behind Rust intent to continue working on Rust. One thing I'm missing, however, is what exactly is considered important to be solved. There are many things how Rust could be made more viable or attractive for a variety of scenarios.
For example, I don't consider it important to be able to compile Rust to JavaScript, because I don't think JavaScript is a good thing.
On the other hand, downstream packaging of Rust and libraries is important. There are scenarios where rustup and cargo are not acceptable. Is this on the radar?
Another issue is target/host support and independent implementations. Rusts only Tier 1 platforms are of the backdoored x86-family. Is this not considered an issue to be solved to make the world a better place?
I don't intent to blame, shame or finger-point anybody, but just point out critical remarks that come to mind.
Splitting out cargo and build systems is something that is explicitly called out and I'm pretty sure there's a bunch of work ongoing to get other Tier 1 targets online.
Everyone has their own opinions as to the work to be done. We now have a yearly roadmap process for each year, and are soon going to be opening that up to community discussion. Feedback from everyone that cares about Rust is extremely important!
That's also part of why I didn't elaborate; my thoughts on that are a whole big post unto itself. That said, my personal focus is on documentation, ease of use, and growth, so those are some broad areas in which I'd like to push hard on next year. The funny thing about goals is that they often tie into other goals, for example, I believe that documentation is a huge part of ease of use. Same with how other people's goals might fit into my goals; language design is also a big part of ease of use, for example.
What I will say is that I'm currently prototyping a complete re-think of how Rust does documentation; I'm excited to share it but it's not ready yet.
To briefly comment on some of the points you've raised:
> I don't consider it important to be able to compile Rust to JavaScript
We share this opinion, but compiling to WebAssembly is a whole different ballgame.
> There are scenarios where rustup and cargo are not acceptable. Is this on the radar?
It's hard to say without knowing more details of what exactly you mean; distros are already packaging Rust and libraries, for example. In general though, this is a thing we've put a ton of effort to in the past, and I expect to have even more in the future.
> Rusts only Tier 1 platforms are of the the backdoored x86-family
A few things; we've been thinking about reconsidering the tier system because it isn't a good representation of support. ARM is basically tier 1 but there's a tiny detail about it that prevents us from fitting it into today's "tier 1" rules.
What we need to improve platform support is experts in those platforms to help us. With our current people, we're doing what we can, but we'd love for people to get involved and drive better support for more platforms.
This stuff absolutely matters; consider Firefox, for example: they have Tier 1 platforms that aren't Tier 1 for us (ARM), but they do depend on us. That's a problem!
> independent implementations
There's one significant independent implementation, but in some sense, this is something we can't actually do ourselves, otherwise it wouldn't be independent. :) I personally don't think this is a massive problem for Rust today; eventually it would be nice though. It does factor into how we do some things, as we try to pretend as though we have alternative implementations, even though we really don't today, in preparation for that future.
>> I don't consider it important to be able to compile Rust to JavaScript
> We share this opinion, but compiling to WebAssembly is a whole different ballgame.
If I understand correctly, WebAssembly is essentially the same as JavaScript, and therefore something I'd like to avoid. I do not want random executable code from websites, that my browser then automatically executes. I don't think sandboxing is a solution, because it's an inherently impossible problem. Externally controlled and unaccaunted executable code is _always_ bad. Instead, let's fix some restricted non-turing complete markup language, like HTML, and stay with that. It's an optimal solution.
Maybe, probably, I'm missing something here, but JavaScript and WebAssembly seem to be things that are not in the interest of users.
>> There are scenarios where rustup and cargo are not acceptable. Is this on the radar?
> It's hard to say without knowing more details of what exactly you mean; distros are already packaging Rust and libraries, for example. In general though, this is a thing we've put a ton of effort to in the past, and I expect to have even more in the future.
Cargo and Rustup seem to be in competition to the existing package management systems. Why not integrate rust and the libraries into the existing packagement systems, like so many other software projects do? How is Cargo and rustup better than Apt?
Also, Apt is perfectly usable without internet. One can download the entire Debian archive to a hard drive and use it without any external connections. How would I do that with Cargo and Rustup? This is just one point. There are so many scenarios Apt already has solutions for, that Cargo and Rustup probably didn't even consider yet.
Anyhow, clearly this issue is noted and even already worked on. That's more I could ask for.
Another thing I forgot to mention is IDE support. I read that Rust is now supposedly supported in Gnome Builder, but I see no option to create a Rust project. This is not the responsibility of the Rust team, and it's just an anecdote, but it affects Rust and it's usage and growth.
> WebAssembly is essentially the same as JavaScript
In some senses, but not others. Regardless of personal feelings though, WebAssembly isn't just happening, it's already happened. And also regardless of personal feelings, many people in the Rust world seem to be very excited about WebAssembly; see https://mgattozzi.com/rust-wasm for example.
(I personally am very interested, but if the community was against it, would accept that. It seems like the majority, at this time, is pretty pro.)
> How is Cargo and rustup better than Apt?
There's a multitude of ways to tackle this problem, but I'll give you the biggest one: I'm a Windows user these days. Apt is 100% useless to me. This problem is the same even on Linuxes, while I do use Debian, I used to use Arch: Apt would be 100% useless to me.
> Apt is perfectly usable without internet.
Cargo is perfectly usable without internet. It's a hard requirement of projects like Firefox, and the various distros that are shipping Rust today. Now, we want to make it even easier to use without internet, so there's work to be done, but the fundamentals already exist and are used in production systems today.
> Another thing I forgot to mention is IDE support.
Yeah, this is super huge, agreed. We've been working on various things, but obviously it depends on what IDE you use, and there's a long tail there. We'll get there!
Thanks :-)
> And also regardless of personal feelings, many people in the Rust world seem to be very excited about WebAssembly
WebAssembly and JavaScript are used to create locked-in software to deliver uncopyable experiences, to ultimately exercise power over other people. Clearly, there will be people interested in this, because the power can be used to generate profit. Ethically, I think it must not be supported. It goes frontally against the idea and intent of free software. I think this problem is glaring, and I'm interested in your view about this very aspect.
> How is Cargo and rustup better than Apt?
I didn't mean to restrict this to Apt. Many projects and programming languages are available on multiple platforms, all while playing nice with the existing package management systems. Apt is not special in this regard, and was just used as an example.
I'm conflicted. I personally waver between "stallman is always right" and "free software misses the point" near constantly, myself. With regards to WebAssembly, it's also complicated. You can argue that JavaScript is basically in the same place, and so for me, it's kind of a moot point. That is, there's no requirement for JavaScript to be free software today either. In practice, obfuscated/minified JS and WebAssembly are basically the same, with the exception that the tooling for de-{obfuscating,compiling} JS is currently more robust and mature since it's been around a lot longer.
TL;DR: I don't think that WebAssembly really changes the game here, for good or for bad.
> I didn't mean to restrict this to Apt
Quite fair! And as I alluded to above, it's not just the thing that I posted. This is quite long but I have little time, so I'm gonna be brief. https://medium.com/@sdboyer/so-you-want-to-write-a-package-m... basically is, IMO, the gold standard for talking about these things. However, I think the distinction of OS/system package manager vs Language package manager is important. Basically, Cargo and Apt (I'll follow your lead here with using Apt as a standin) have very different goals, and very different constants. This is because they're doing two very different jobs. Specifically, system package managers are about providing a unified set of packages to end-users. Language package managers are for developers to build applications.
With Cargo, we've very strongly stayed away from giving it any sort of "system package manager" features, because that's not what we're trying to do. However, at some point, a developer delivers an application to an end user, and so I agree strongly that Cargo should play well with Apt. It cannot, however, be replaced by Apt. To that end, I think tools like https://crates.io/crates/debcargo are the path forward, and in fact that tool is (in my understanding) being used by Debian to manage this bridge. I would love to have a simple Cargo command to generate a compatible system package for any of these systems. It's a long tail thing though, so it's gonna take some time.
Anyway, yeah, that's the root of it. There's tons of other details too. Package management is a very hard thing. Maybe someday I'll write a blog post about all of this at length, when I have the time.
> Many projects and programming languages are available on multiple platforms, all while playing nice with the existing package management systems
What's the gold standard example for you, exempting C and C++?
> TL;DR: I don't think that WebAssembly really changes the game here, for good or for bad.
It's a step in the wrong direction. Working on WebAssembly is not as bad as it could be because there already is JavaScript, but instead of working on WebAssembly, I think we should bury JavaScript.
> Package management is a very hard thing. Maybe someday I'll write a blog post about all of this at length, when I have the time.
Fair enough. Looking forward to it.
> What's the gold standard example for you, exempting C and C++?
Maybe Python on Debian. But why shouldn't C be the benchmark? And why compare to another language instead of what's desireable?
Cool, thanks. I'll look into it. I'm more familiar with Ruby and Debian which has a... storied history :)
> But why shouldn't C be the benchmark?
In some sense, it shouldn't, I'm in agreement that in some sense, we should talk about what's desireable. However, there's a reason that it's significant: apt is designed around C. Its compilation model, everything. So of course C works with apt very well; I'd hope so! But non-C languages have a much larger hurdle. In terms of Rust's integration into a system, it'd face challenges similar to non-C than to C.
It would be nice to have a list of what each of Apt, Pacman and Portage needed to change to accommodate Rusts needs.
Thanks for your comments!
Someday I might write some of this up, but honestly, I deeply appreciate the work that the humans who build OS packages do, and don't really want to add more friction to the pile, or appear to be denigrating their work. It's more likely to just be an "argue on the internet" situation than actually cause any change that'd be helpful.
To be clear, again, this stuff already works. It's about the difference between "easy" and "possible".
One thing that might help people see this distinction better is if Cargo offered better post-build hooks to allow community-driven add-ons that handle packaging for the various system package managers. If a Rust developer could run cargo dist-package (or whatever it's called) and have it build a deb/rpm/pkg/msi, it'd be really easy to use cargo for the development phase and the other package managers for the distribution phase. As it is, if I want to run someone else's Rust software, I'm basically using cargo install to pull from crates and build locally.
We've been wary to add more hooks because then you can start to emulate a system package manager with them.
But 'hooks' also means making cargo more extensible. IIRC, cargo will error out if it finds a key in Cargo.toml that it doesn't recognize. It would be nice to be able to store configurations for extensions to the cargo build lifecycle in that file as well. Cargo could just ignore them. Also, it'd be nice to have a crate to read and safely write from/to Cargo.toml to make it easy for extension executables like you mentioned to use information about the build.
Just being able to pass through to an executable in $PATH isn't really enabling much and, aside from that, cargo seems pretty hostile towards implementing features that the team doesn't believe in. I guess what I'm advocating for is providing generic extension points in Cargo and then sitting back to watch what the community comes up with. My experience with the Rust community is that there are a lot of exceptionally smart developers who love the language and the tooling and I think we'd get a lot of exciting and creative features if the main team were removed from the critical path. A more organic approach to evolving the way we build Rust code might come up with something better and can serve as free user research for what should be included in the official cargo tool and become the officially-recommended best practice.
I haven't heard the phrase, "free software misses the point", before, and a quick web search turns up too many links to stallman's "why open source misses the point" article. Could you elaborate a little or link somewhere that does?
Thank you!
I made that phrase up. But I'm sympathetic to the essay, and I think maybe I was subconsciously referring to it.
Okay so, my issue with free software is similar to my issue with liberalism. I mean that in a philosophical, historical sense, not a "the Democrats" sense.
Liberalism (and much thought from the Enlightenment) is focused on freedom with regards to exchange. The Four Freedoms are focused on "I have this artifact, what can I do with it in relation to others." Marxism, which I am more sympathetic to, instead focuses on production over exchange. That is, the issue is not in individuals exchanging things, but how those things are produced in the first place. So in some sense, it's pre-occupied with the wrong thing.
Let's make this more concrete. You know the concept of "source dumps"? Let's say that I'm MegaEvil Inc. I can create some software, and once a year, release a new version. I also give you a copy of the source code, licensed under the GPL. There's no mailing list, no way to send patches, no governance. Just a copy of the source, once a year. This is 100% Free Software, but I'd argue that it misses the point. It's still better than being proprietary! But it doesn't let me, as the user, exercise any control over how the software is made. Yes, I could create a fork, but what happens when one year, there's an extensive internal re-write? At some point, a fork is a completely separate project, and we're back to step 1.
That is, the question I'm interested in is, who has the power to affect the development of the software? Free Software has nothing to say about this. But that's the real step forward. Being at the complete whim of upstream isn't that different from being at the whim of a proprietary vendor.
Given that I've literally just thought of this, this might be a bit rambly, a bit too deep, and a bit under-developed. But there you go. :)
I suppose I could sum it up as: "GPL doesn't include anything about the infrastructure of the software." The lack of an infrastructure for developers to hook into, could be GPL in literal word (or law), but not GPL in spirit (or morals). Likewise, and to add to your example, client-server architecture such as cloud computing might be GPL compatible in literal word, but not in spirit.
Can I suggest a compromise with the belief that "Stallman is almost always right," because no human being with an extremist ideology is ever always right. A philosophy can be useful, even beneficial, but still be flawed.
> A philosophy can be useful, even beneficial, but still be flawed.
I agree wholeheartedly.
> Also, Apt is perfectly usable without internet. One can download the entire Debian archive to a hard drive and use it without any external connections. How would I do that with Cargo and Rustup? This is just one point. There are so many scenarios Apt already has solutions for, that Cargo and Rustup probably didn't even consider yet.
You can't simply ignore 98% of the users of other operating systems just because it happens for you today to use a Debian based system. The fact that Rust has an OS independent package manager is a huge plus for Rust adoption.
I agree that an OS independent package manager is a plus for Rust adoption. I also think it cannot replace the OS package managers, and therefore Rust should also play nice with them, and integrate as much as possible.
This makes me curious: can you tell me some examples of other software projects which are not written in C that integrate libraries into existing package management systems?
Personally, I feel like `apt` could almost be called a C/C++ package manager at this point. It does have some packages written in other languages, but most languages still have their own package manager for development. For example:
* Python: has pip and virtualenv, packaging with apt is difficult due to incompatible version requirements * Ruby: not personally using ruby much, but seems like most apps are installed via bundler or gem and not packaged * Haskell: Small binary compatibility of libraries make traditional packages difficult (even dependencies of dependencies need to match for compatibility) * Java: I don't even know how java libraries would work with apt * PHP: I have never used a PHP package installed through the package manager
Granted, I may be missing something here, but the majority of the packages in apt repository seem to be C/C++, maybe some perl or python which are "made to work" simply because too many things depend on them.
> As such, with hundreds of Python modules and multiple versions of Python supported, Debian is the largest "integrated Python distribution". (https://wiki.debian.org/Python)
Even if other languages don't play as nice as possible, or even foul, that doesn't mean it's not important for Rust to play nice.
Agreed that it looks like re-inventing the wheel from the outside; rest assured, it's useful!
The Rust project does not have a goal to destroy every single piece of technology that could be construed as user hostile. The reality is that if you want to write a widely used browser application, you need to use Javascript, and probably soon, optionally WebAssembly. Rust is well positioned to take advantage of that.
(This is not an argument against the claim that Javascript/WebAssembly are user hostile. I have no real interest in discussing that, and frankly, there is no obvious conclusion to draw there. Instead, I'm pointing out a category error.)
what do you mean, exactly?
people are interested in compilation to javascript principally because js is the language of the browser; if it was just another scripting language (like perl, ruby, lua, etc) there wouldn't have been such a transpiler explosion.
Edit: Those who downvoted, do you disagree, or was there another reason?
Javascript can be turned off, or modified at will in the browser, and its source is available and viewable. Most Javascript is FOSS licensed, and can be forked, edited and redistributed. Minified libraries usually have plaintext sources accompanying them, meaning they can be studied and validated. WebAssembly, meanwhile is barely being used at all, much less in any way that can be considered 'like DRM on steroids.'
I understand a lot of people here don't like Javascript, and dislike WebAssembly even less for being a binary format, yet as languages and their environments go, Javascript makes itself far more amenable to user freedom than others, and WebAssembly still barely even exists.
Seems to be confusion here between "I'm not interested in using that technology" and "Nobody should use that technology".
If you're saying the first, fine. I often feel that way too, it's not that important a statement since you can just not use it and move on.
Someone has to do Javascript though: targeting the web browser client is important and here to stay although it is evolving, and wasm is part of that evolution.
If you're saying the second then that's not in line with existing reality. The ship has sailed on whether we're going to have apps running in the browser or not.
WebAssembly has nothing to do with Javascript, though. I don't think that's what the GP means.
They are different VMs in the browser alongside each other, I was wrong about that (1)
However, this comment says more about the ways that they are and aren't equivalent (2)
Both are for "targeting the web browser client" so they have that in common, and a lot of the logic about their place in the world is therefore the same for both of them.
My understanding is that this is not true; wasm is more of a particular mode or API of interacting with the existing JS VM rather than having its own. You want to re-use the JIT, etc, for wasm. Note that that issue is from 2015.
That said I don't work on these implementations so maybe I'm wrong as well.
https://www.viatech.com/en/silicon/processors/quadcore-e-ser...
There was also another article indicating they have a 28nm chip in works with better performance. They're designed by Centaur Technology in Austin, TX. Might not be immune to national security activities in that jurisdiction. They'll also be way slower than desktop-class chips of Intel and AMD. Yet, there are options for x86 past Intel and AMD. Like everything else, the odds of a backdoor from groups like NSA also gets lower the older the design you buy since most were focusing on software and crypto back then.
EDIT to add link below. Anyone interested in hardware design, esp FOSS, might find their server farm impressive or daunting depending on how cheap you wanted CPU design to be. ;)
It won't be easy to find a project in Rust or to get a customer to start a project in it, but I sure would like to.
Most of what we're seeing in the "Rust jobs" space is companies hiring people to do Rust work, but also other work, and so they're not pure "Rust jobs". For example, take Dropbox, one of our oldest production users. They have a job opening for a Senior Software Engineer right now: https://www.dropbox.com/jobs/listing/735139
Nowhere in this listing do they mention any specific technology. Will you get to use Rust in this job? Maybe! Maybe not. They tend to hire good programmers, and expect them to be flexible, in my understanding.
Another genre is stuff like Amazon, who used to have a job posting up here, that now 404s: https://www.amazon.jobs/en/jobs/559813/sr-software-developme...
This posting said that knowing Rust was a plus for the job. Does that mean you get to write Rust? I can't say, as I don't know the details of what that job entails, but it might be a good way to differentiate yourself from the crowd.
Anyway, I think jobs using Rust are extremely important, and want to see more of them. It takes time though :)
I do, but with some caveats. Rust is pretty great at spitting out JSON, so I think it fits in best as either the backend for an SPA, "middle end" style services, and/or background jobs that need to do intense processing.
One area that's under-appreciated about Rust in the web space is its low memory footprint: crates.io is written in Rust, and uses about 30MB of RAM, resident, all the time. Lower memory usage means saving $$$ on servers. At least one production user has reported being very happy with this.
Second, npm has stated that they enjoy Rust for their middle end stuff because it's "operationally boring". The Rust service (and soon to be services) is never what triggers pager duty. It just does its job.
We already have decent languages for the backend - if you like types, there are various languages on the JVM, and if you don't, there's Go. They use a lot more memory than Rust, but not a deal-breaking amount for most people, and they go about as fast. They're as or more productive, and likely to stay that way, because of garbage collection.
What we don't have is a decent language for desktop applications, native mobile apps, operating systems, high-throughput low-profile infrastructure, embedded systems, and maximum-performance work. We have C/C++. My life, as a user and a programmer, would be improved much more by Rust levelling up and taking over that space than by getting into a HTTP-pissing match with Kotlin and Go.
Although HTTP does matter outside web development as well. For example, we need a good gRPC library for Rust.
But thanks, I'll keep an eye on it.
I don't wanna spam comments with just "<3", so please consider this a collective reply.
I can see how some may have read some of this as me leaving, but I can assure you, I'm not. I'm more committed than ever.
That said, I understand this sentiment, and it's why I didn't submit it myself. I think it's fine, as people do seem to be interested, but can also understand the opposite view.
It’s an interesting facet of the human experience that lets me live Steve’s life for a moment through my own lens of life experiences.
Thank you!
Either way, your help in providing documentation and mentorship to me personally on IRC and thousands of other aspiring rust devs is something I'm profoundly grateful for. Thank you, Steve!!