Rust 1.40
blog.rust-lang.org
blog.rust-lang.org
I do a fair amount of integration with C, and the work the rust team is doing for procedural macros has helped a lot. Being able to use them in an extern blocks will surely help more.
Like many, I have a wish list that I complain about sometimes, but I feel bad about complaining because rust has done so much to improve my use case I don't know where to begin. I should write up a blog post about all of the difficulty I had, and how each release since around 1.28 has introduced features that solved these problems one by one.
To me, Rust/C interop is the killer feature. Not only is it a low-level language that can be used instead of C, it can also be used to extend C applications. I'm sure it was not easy to do this well, but what a great strategic decision by the rust team to spend the effort on it!
Release notes are helpful -- when you see something added related to C, it's a strong hint to read about the new feature.
Macro features are important because rust has sophisticated macros and C macros are closer to text replacement. So when trying to emulate a C header, you need to do a lot of macro magic in rust sometimes.
Sorry that I don't have more to offer from personal experience.
Here's the one for C:
When a new state is added to an enum, we want the code to not compile so that we can fix all the places that need updating.
Yes, some enums are intended to be expanded. Still, as long as something is a compile-time error, I don't care about having to update my code when I choose to upgrade compilers.
In fact, even if the enum is intended to be expanded, I prefer to hear about new features and changes on APIs I actually use in a given project. It allows me to review mu choices and many times in complex APIs there are new options or flags that are useful to know about and otherwise get ignored. It allows me to update all callers as needed.
I have heard too many times the backwards compatibility story in the C++ world and I never valued it for things that are errors rather than behavior changes. It makes the libraries and the language way too rigid and makes evolving it a pain.
As long as you don't change the meaning of code that compiles cleanly, feel free to change things.
If you really want to keep code compilable because you are doing a chance that you believe will be a PITA, you could always offer a (truly) automated tool.
It's not just your code; it could be the code of any of your dependencies (or their dependencies, recursively). Besides, the Rust team does intend to keep older code compiling unchanged on newer releases of the Rust compiler, unless the breakage is caused by fixing a soundness hole.
The best example of a non-exhaustive enum, in my opinion, would be std::io::ErrorKind (https://doc.rust-lang.org/std/io/enum.ErrorKind.html). Several Rust releases ago, I added a new variant to that enum (the last one, UnexpectedEof, used by std::io::Read::read_exact - notice how that variant comes after "Other", which used to be the last one). If that enum were not non-exhaustive (non-exhaustive enums already existed since Rust 1.0, though using a doc-hidden trick instead of formal compiler support), I would not have been able to add that new variant, since it would risk breaking anything which matched on every variant of that enum.
For example suppose you add another image format to the `image` library:
https://docs.rs/image/0.23.0-preview.0/image/enum.ImageForma...
What's more annoying - a major version bump every time you support a new format, or not being able to exhaustively match on all 10 formats?
However sometimes you are a dependency and you want to give up this restriction to gain the ability to add things without having to bump your major version number.
By far the most common example is error enums which don't necessarily need all of their downstream crates to handle every error, they likely are bucketing most of them anyway and non_exhaustive ensures they support that.
This is a decision library users should be making, not library writers.
If you want to do that, you could edit your own copy of the library's source code and nobody will mind. Maybe that's enough?
It seems like you shouldn't be able to publish a crate where you're using an upstream library in a way they don't consent to, because now you're involving others in this dispute. A basic requirement for publishing to a shared open source ecosystem should be that you're resolving any disputes you have with upstream libraries and not just going your own way.
Of course, the trade-off is obviously that by choosing to make things continue to compile when something changes, you are no longer causing things to fail to compile when something changes. I'm uncertain how this tension will be resolved in the long run.
(Context: used to do iOS dev w/ obj-c; haven't used Swift, but keeping an eye on it, mostly out of curiosity.)
There is the Xcode migration which auto updates source code to the new version, which I’m sure you are aware of and leaves you annoyed. Thou it’s more than “no effort”
Congrats to everyone who has been contributing to Rust, it's definitely moving the language forward!
The progress is still cool though.
It's pretty difficult to use AWS without touching rust code, for example, and impossible in the case of Dropbox.
#[non_exhaustive]
https://www.rust-lang.org/production/users
Jumping into rust for our major projects doesn't make sense. Existing code is largely C, with some assembly and C++, so rust would just add another language to the requirements.
To somebody who isn't already a rust expert, it isn't even clear that rust is good for the job. We like bitfields as L-values in C, without ugly macros or other wrappers. We like to be able to easily produce a small relocatable blob of binary executable code, such as a firmware image or boot loader. Speed and size matter to us. Sometimes we compile for weird targets such as ARMv4, Coldfire, 80286, and Xtensa.
C certainly isn't perfect. If rust had fewer limitations, that could be convincing. My wishlist is probably zero-priority, but FWIW: I'd like pass-by-value arrays, naked functions, easy ability to get things in place as desired (position independent or fixed) without a PLT or GOT, portable bitfield layout, specified calling conventions, computed goto, and other things that high-level programmers despise.
Plus there isn't really anything else in the market that directly competes with it outside of what it tries to replace (C/C++).
For example: https://msrc-blog.microsoft.com/2019/11/07/using-rust-in-win...
I'd call D a dead language. Can a dead language compete with a growing one?
I'm on a Discord for Dlang that's quite active honestly. The D community is plenty active. Their forums and IRC. Maybe not as active as other language communities, but it's not made by Mozilla or Google which gets a lot more attention.
The already "use a different language" for most of their stuff.
Their D use is tokenish, the way you can find any bizarro, niche language used somewhere big. That doesn't mean the language is in any kind of widespread use, just that some teams in some big company or another adopted it - as outliers.
Source: I worked at Facebook for four years. My first diff at Facebook was even in D, in a project that was already effectively abandoned by the time I changed it.
It's really a tiny, insignificant minority of actively running code there, possibly zero by now.
https://github.com/benfred/github-analysis/#inferring-langua...
Rust by contrast is 16th and trending upwards year on year. So these D users that are missing from internet forums are also missing from checking in actual code.
You talk about the lack of facts, but I can link you to Stackoverflow's survey, Github's usage, Google Trends, and show you that D isn't doing very well. Where's your facts here? Where is this evidence of a large number of quiet D programmers?
So let's talk facts, I have, your turn.
I'm not sure how a language that is in decline, and hasn't grown much in its life, isn't a dying language (I'd argue dead at under 1% after that long).
> I'm not interested in being right on HN.
So you show up, call out other posters for having a "factless debate," to which they respond with facts. They then ask you for the same and instead of providing them to prove your point, you're suddenly not interested in a discussion on HN? That seems like bad faith posting to me.
It is competing for my attention and many others I guess.
> I'd call D a dead language. Can a dead language compete with a growing one?
I'm much more inclined to spend time with D than with a number of other popular languages if I could choose.
I don't know if I'm alone in this, but for what I know kt might become really popular in the future. (Look to Erlang for an example of a language that went unnoticed for a couple of decades or so.)
FWIW Rust is on top of my list of languages I want to master if I should get time.
The other major competitors to Rust is Zig and Jai. Of course Jai has the problem of being unreleased at the moment (and its fate is tightly bound to Jon Blow).
There are also a few other languages attempting to make inroads in the space. For example, Odin and Kit.
Here's my take on it though. C and C++ will always have some sort of systemic problem because they are able to do too much. In order to be a system language AND also do everything that people want a system language to do (games, embedded, high performance) you need (I assert) to have rough edges and dangerous pit falls.
Rust will eventually beat C/C++ on making web browsers and similar technologies because that's what it was built to do. However, Rust probably won't be able to beat C/C++ in game development and total OS development (although it can probably be partially used for both). Enter Zig and Jai.
D and Go both compete with C/C++ in a space that was temporarily taken over to Java/C#. But in the long run may end up being ceded to something that's less than a managed language but more than a system language.
Ultimately I predict we'll see C/C++ slowly give way to a family of system languages that all hold different niches before finally becoming a relegated to legacy only. This could still take a few decades to complete. And if you look at newer versions of C++ it's possible that C and C++ may even evolve to hold a different niche than the wide series of domains that they used to hold onto so tightly.
"In order to be a system language AND also do everything that people want a system language to do (games, embedded, high performance) you need (I assert) to have rough edges and dangerous pit falls.
"Rust will eventually beat C/C++ on making web browsers and similar technologies because that's what it was built to do. However, Rust probably won't be able to beat C/C++ in game development and total OS development (although it can probably be partially used for both)."
Rust has all the necessary escape hatches (through unsafe) required for these spaces. There are people working in these spaces with Rust successfully, today. So, while, the other languages you mention might find success here as well, there is no reason (from a technical perspective) that Rust will not.
Rust will always leave plenty of room for C++ to the extent that it tacitly encourages suboptimal software architecture for some types of applications, such as database engines, that commonly rely on safety models Rust was not designed to express.
I do see Rust potentially replacing a lot of backend Java, eventually.
IIRC while Rust does allow you to switch up allocators, it doesn't let you mix and match your runtime with multiple allocators. (Each artifact can be linked with at most one allocator at a time.) There are applications where you want to have multiple allocation strategies for performance reasons.
Uniqueness is really useful for most applications, but there are times that you want data structures that allow multiple pointers to the same data. Having to do this in Rust is going to be a bigger chore than doing it in a language that doesn't support uniqueness.
It will be possible for Rust to still participate in these areas (especially because you can use Rust for only part of your project, so you can use it where you don't have allocation or nonuniqueness constraints in your problem space). However, other options are going to offer a better programmer experience.
So, sort of yes, and sort of no. Like, you can swap the global allocator, but there's no way to parameterize standard library stuff over anything but the given allocator. But for your own code, you can write and use allocators however you want. Arenas are often popular, for example.
Do you think you could expand on this?
From my experience Rust facilitates all the same operations as either C or C++, and generally without even needing to turn to unsafe. What I've found in my own (not DB, but networking) work, is that Rust generally asks you to restate the problem in a way that will allow it to be best expressed in Rust. This often differs from the down the middle of the road implementations people have grown used to in other languages, but it doesn't in any meaningful way prevent you from solving the problem, in a safe way.
Rust assumes that all references to memory are visible at compile-time, and the safety analysis can be applied in cases where this is true with the usual caveats around borrow-checking. The "unsafe" facility is designed to interface with code written in other languages that don't respect Rust's model, and it works well for that. But how do you express the case, common in database engines (because direct storage-backed), where hardware can hold mutable references to most of your address space? There is no way to determine at compile-time if a mutable reference will be unique at runtime or to even sandbox it to a small bit of code. As a consequence, most memory reference are effectively mutable. There are workarounds that will minimize the quantity of unsafe code in Rust if you are willing to sacrifice performance and elegance.
In databases, having many mutable references to the same memory has few safety implications because ownership of memory is dynamically assigned at runtime by a scheduler that guarantees safe access without locking or blocking. This safety model solves the hardware ownership problem, which is why it is used, but it also enables quite a bit of dynamic optimization even if all your references are in software so you'd want to do things this way anyway. In C++, you can make all of this largely transparent on top of explicitly mutable references to memory. Again, you can produce a minimally unsafe version of this in Rust but it is going to be significantly uglier and slower.
As more server software moves to userspace I/O and scheduling models (for performance and scale reasons) it will be interesting to see how this impedance mismatch problem is addressed in Rust.
Consider a SQL database engine.
Many threads/process can be accessing a SQL database at any given time.
But if two of those actors require write access to a same object, only one will be allowed access and all others are blocked.
So at this high level the concurrency is guaranteed by the engine, which then means at the low level the engine can safely assume the access is exclusive.
As for hardware DMA’able memory, it’s true that it adds friction to work with in Rust. But C or C++ would fall into the same boat - one would need to sprinkle volatile or atomics, as appropriate, to avoid the optimizer from interfering. In Rust, you’d need to do the same (ptr::{read,write}_volatile or its atomics).
I’m having a slightly hard time imagining a db where “most” of the address space is DMA’able. I’ve some experience with kernel bypass networking, which has its own NIC interactions wrt memory buffers, but applications built on top have plenty of heap that’s unshared with hardware. What’s an example db where most of the VA is accessible to hardware “arbitrarily”?
Also, regardless of how much VA is shared, there’s going to be some protocol that the software and hardware use to coordinate. The interesting bit here is whether Rust and its type system can allow for expressing these protocols such that violations are compile-time detectable (if not all, perhaps some). Any sane C++ code would similarly try to build some abstractions around this, but how well things can be encoded is up for debate.
When a typical Rust discussion ensues, it’s commonly implied (or occasionally made explicit) that “write X in Rust” == “write X in safe Rust”. And this is the right default. But I think any non-trivial system hyper optimized for performance will have a healthy amount of unsafe code. The more interesting question, to me at least, is how well can that unsafety (and the “hidden”-from-rustc protocol) be contained.
As for a db scheduler obviating the need for a compiler to arbitrate ownership, that’s certainly true to a degree. But, this comes back to the protocol I mentioned - the scheduler is what provided the protocol and so long as other components work via it, it can provide safety barriers (and allow for optimizations). But again, I don’t immediately see why Rust (with careful use of unsafe) couldn’t do the same. And afterall, everything is safe so long as things play by the (often unwritten or poorly so) rules. Once systems get big and hairy, it gets tougher to stay within the guardrails and that’s where getting assistance from your language/compiler can be very helpful.
Some of the Rust libs/frameworks for embedded/microcontrollers deal with hardware accessible memory and otherwise “unsafe” protocols, but I’ve seen some clever ways folks encode this using Rust’s type system.
- All pointers are ordinary, the fact that the same memory can also be DMA-ed by the hardware is immaterial. You do need accounting mechanisms that let the code infer which objects in memory are at risk of being read/written by a DMA operation. No atomics or volatiles required in userspace. Modern database code is effectively single-threaded.
- Most of the address space in a database is DMA-able because most runtime structures in a database engine must be adaptively persistable to storage. There are various workloads that will force different parts of your runtime state to be pageable because they can overflow RAM while operating within design constraints. Unless you are assuming small databases, that complexity is inconvenient but necessary for robust systems.
- C++ is more expressive than Rust when it comes to making hackery like this transparent, most of which is resolved at compile-time in C++. Much of the mechanics can be taken care of with a pointer wrapper that heavily overlaps the semantics of std::unique_ptr, making the code quite clean and natural. Most code never needs to know how that magic happens. C++ compile-time facilities are currently far beyond Rust.
- You can formally verify the scheduler design, and we sometimes do, but actually implementing it efficiently in real code without the borrow-checker losing its mind is a separate concern.
As I originally stated, you can write such systems in Rust while managing the amount of unsafe code. You just wouldn't want to and there would be little to recommend it compared to the C++ equivalent since it would be objectively worse by most metrics.
The volatile I mentioned isn’t due to concurrency of userspace threads, but to avoid the optimizer from eliminating read/write operations. If the src/dst of those memory ops is DMA’d memory touched by hardware, you’d need to do that. Has nothing to do with concurrency.
Capability to spill to disk is certainly needed, no argument. But “most” of the address space and “most” of the runtime structures? Can you elaborate? Is there an OSS example or some paper or any discussion of such a thing in the open?
You can have custom smart pointers in Rust just as well, and back them with your own mem allocator. While there are features in C++ not currently available in Rust, C++ facilities “far beyond” is hyperbole. How well do you know Rust? Genuine question.
I'm not quite so sure. Rust can be a good fit for services that need to be very optimized for memory or CPU usage. Java brings so many other extremely important benefits like introspection, management, profiling, hot swapping, and being generally more productive due to the fact that it's a GC'd language. Not to mention the huge ecosystem behind it.
I call this general concept Brute-Force Assurance where you just modify the form of a program to fit existing tools to get their benefits. Just throw every sound and/or complete analyzer plus a lot of test generators at it. Also, code in a way that helps those tools wherever possible. If one can't, then use them on a version designed for verification first to get the algorithm right, step it toward optimized version, equivalence tests, repeat, etc.
* Just because I'm more comfortable with types doesn't mean that everyone else is.
* Someone may want to do something in a type system which is well typed, but only in a different type system.
* Someone may want to do something in a a type system which is well typed, but which has some bad compilation characteristics for the given type system.
Even if Rust is objectively better, you still have to get used to the things about it that make it objectively better. And you have to keep up with the changes that are made to it. And you have to understand where those better things fail down (for example Non Lexical Lifetimes ... in which case you have to get used to the NLL acronym that people use).
A simpler language, even if objectively worse, can yield better results if it is used with discipline. Discipline that might be easier to hone with less things that need to be considered.
And sometimes better results don't actually matter because the goal isn't the best results tomorrow but reasonably adequate results today.
http://archive.adaic.com/intro/ada-vs-c/cada_art.pdf
I'll also add that Rust can give you both memory safety and race freedom at compile time. If you debugged heisenbugs, then you know that's a huge benefit. On Lobsters, one guy mentioned being hired for (a year?) to find and fix one in a system. Eiffel's SCOOP had a similar benefit. Languages such as Chapel made parallelism super easy in many forms vs C++ and MPI. Used judiciously, macros can eliminate tons of boilerplate. Erlang's strategy for error handling might go in this list if reliability is a goal.
There's been quite a few examples were a difference choice in language design eliminates entire classes of problems with anything from no effort to significant effort by developer. Increased velocity with fewer bugs during feature integrations and maintenance are provably-beneficial metrics for a business. I think we can say there's scientific evidence of actual benefits from language choices which have potential benefits if used in business. I just can't tell how far, if any, you'll get ahead by using them since there's non-language-design factors to consider that might dominate.
The large uptake and excitement around Rust shows that there are many C and C++ programmers who appreciate the safety guarantees that it provides. The popularity of Rust has actually created a resurgence of interest in Ada and each language has benefitted from the other.
For example, Spark, a well-defined subset of the Ada language intended for formal verification of mission-critical software, is adopting safe pointers that were inspired by Rust (source: https://blog.adacore.com/using-pointers-in-spark).
I would not be surprised if Rust also adds features based on ideas from Ada.
This is good. The "fast and loose" qualities of C and C++ allow far too many errors and security vulnerabilities in software. We have better tools. We just need to use them.
Garbage collectors provide memory safety without any special syntax.
Or in Delphi all strings are reference counted. They are mutable, if the ref count is 1, and immutable when it is larger than one. It is memory safe with safe aliasing and needs no special syntax.
A sufficient smart compiler could just optimize the reference counting away, and treat everything as immutable, unless it can prove there is no aliasing. Ideally, a language would only have two kinds of reference, mutable and constant, and everything is figured out by the compiler
I guess Box<T> is special-typed but at the surface it just looks like a normal type
Yes, there's also a C pointer type, but it's for interop
But getting the C committee to act is about as hard as designing a new PL.
Now, also, will this new SaferC also bring with it any of the other features people appreciate in Rust? Such as data race free code (because of the strong type/trait system and Send/Sync auto types), or match and let statements that support destructuring of types through pattern matching, or monomorphism for zero overhead polymorphism, or the simple to use tools around the language for managing dependencies, or async programming model that strips away all the complexity of hand written state machines?
For me, all of those features make Rust a modern 2020 language. I’m curious what a SaferC would have. And frankly, if it could exist, why hasn’t it been developed in the last 50 years?
But, let’s say you’re right. D is SaferC—are the folks who are still waiting for a SaferC able to recognize it as such? Or, have they already decided that like Rust it doesn’t meet the criteria of the language that they’re waiting for?
Go established itself as a python/ruby/php competitor. Rust is a c/c++ competitor
Imagine one second Kubernetes, Docker or Prometheus those app have nothing to do with PHP / Ruby. It's related to C++ / Java.
Prometheus is the equivalent of what's at Google and it was built in C++. Same for all the DBs outhere in Go: CockroachDB, NATS, etcd ect ... those are C++ applications related.
However you are talking about projects that are not particularly language dependent as long as the runtime is reasonably performant. They could just as well have been implemented in Java. For many Rust would arguably be a better choice but that is not relevant. Go shines here, but Rust is barely emerging in these areas.
Rust is a C++ competitor in areas where C++ has no competition because a GC is a no go. This is also where the most prominent use cases are.
Most of the fields where rust is making headway, it is making headway for precisely the things that Go is not known for - lack of a garbage collector, strict and expressive type system, robust generics, etc.
https://en.m.wikipedia.org/wiki/Rust_(programming_language)
10 years is a long time. Swift is 5 years old and it seems old and baked.
Considering the buzz Rust has had on HN, etc, I thought it was more popular.
Sounds to me like it needs some sort of push to get over a tipping point.
Often part of the problem isn’t technical. In business people don’t want to be first. Perhaps a few more high-profile users.
Besides a big target for Rust is the C and C++ world. On Hacker News in my experience you have a majority of commenters coming from the web world where no news in one year effectively means that the project is dead.
For us in the low level world "stable for 4 years" means "maybe we can start considering using it in production" and the lack of big buzz every other month is more a pro than a con. I'll take boring and reliable over shiny and breaks-every-other-year.
I've just started adding a new feature on a C project started in 2009. If I used Rust I'd want to know that my dev environment will still be usable in 2019. I think the commitment to stability will pay off eventually.
My personal intuition is that it will have become a strong alternative to C++ by then, and Go will probably eat the other side of that (at the 'upwards' frontier of C++, before/underneath e.g. Python), which given a decade could result in maybe 25-30% of major C++ projects moving or with plans to Rust/Go. That would be a healthy balance of alternatives, a true victory for these mid/low-level contenders.
It's not like the bottom of the stack can be won the likes of Python or Js at 80-90% within a decade. Structurally, it simply cannot.
To me, Rust "feels" like one of those languages that will stealthily become very important at least in certain segments where security, performance, C ABI compatibility (dll/so/dylib) and zero runtime are required.
I don't see your point?
> Rust "feels" like one of those languages that will stealthily become very important
Rust is not very stealthy at all :)
My point is that C++, Java, etc. got established long before the hiring patterns and keywords changed. The employers will just suddenly start to expect (language age + 1 year) experience.
I think same will happen to Rust.
> Rust is not very stealthy at all :)
Yet it might appear like that from corporate point of view.
I don't think this is necessary. The attribute in question is applied to an enum variant, and that variant's constructor is then given only crate-wide visibility. This looks to be simply a compiler-enforced codification of the pattern you're describing.
> A perhaps more important aspect of #[non_exhaustive] is that it can also be attached to enums themselves. An example, taken from the standard library, is Ordering
For example, take this _Nonexhaustive enum case here: https://github.com/est31/rcgen/blob/d6b84d3d9d51b088dd672975...
A little bit further down, I'm matching on the enum in the to_oid function. I'd prefer if I got an error or at least a warning pointing to the match if I added a new enum case and didn't update the match statement.
[0]: https://play.rust-lang.org/?version=stable&mode=debug&editio...
Thanks for sharing!
For anyone else who hasn't: https://rust-analyzer.github.io/ and https://ferrous-systems.com/blog/rust-analyzer-2019/
But the experience is vastly better. I deleted RLS immediately. And it's not that hard to install either.
https://rust-lang.github.io/compiler-team/working-groups/rls...
Instead, this is what will be elegant:
pub enum SearchOption {
Left,
Full,
}
pub enum SearchOptionBeta:SearchOption {
Rigth,
}
This mean MATCH touch "SearchOption" as the "stable" API and the inner crate touch SearchOptionBeta. When SearchOptionBeta need magic is that it match SearchOption too. Later is just a matter of replace one for the other.
And this is even useful for more than to stabilize certain fields or arms...
pub enum SearchOption {
Left,
Full,
Beta(SearchOptionBeta),
}
#[non_exhaustive]
pub enum SearchOptionBeta {
Right
}If enums were abled to be extended:
match e: SearchOptionBeta {
SearchOptionBeta:: Left, Rigth, Full
}Of course it's an error, as Bar doesn't have a field called ,,a''