MiniRust
ralfj.de
ralfj.de
"MiniRust is the cornerstone of my vision for a normative specification of Rust semantics. It is an idealized MIR-like language with the purpose of serving as a "core language" of Rust. This is part of a larger story whose goal is to precisely specify the operational behavior of Rust, i.e., the possible behaviors that a Rust program might have when being executed: the behavior of a Rust program is defined by first translating it to MiniRust (which is outside the scope of this repository), and then considering the possible behaviors of the MiniRust program as specified in this document. That translation does a lot of work; for example, traits and pattern matching are basically gone on the level of MiniRust. On the other hand, MiniRust is concerned a lot with details such as the exact evaluation order, data representations, and precisely what is and is not Undefined Behavior."
It is yet another step on the path to having a Rust specification, especially with respect to precisely specifying the capabilities and requirements of `unsafe` code.
Zig is also very compiler dependent, but at least the language has direct enough translation as a goal, that it's straightforward to learn the semantics from trial and error alone.
It wasn't standardized until 1989 - 17 years later!
I think Rust will do just fine, and the compiler error message is (IMO) the best of any language, making adhoc exploration possible.
Can you imagine trying to explore C from ad-hoc alone based on its compiler errors?!
It’s in many ways similar to the way the early web was developed, different browsers tried out many different additions, copied the better ones and improved them. The standardisation process we now have didn’t exist back then, and has been through a number of iterations itself since (the “xml every thing” process that ended with xhtml had a questionable outcome at best).
Some may argue that a formal standardisation process should have started earlier, before IE became too dominant. But I’m not too sure, IE was incredibly innovative and introduced some important technologies and apis, before stagnating.
In fact the “modern” html/css/js standardisation process is specifically lead by individual browsers implementing and trying out new additions, with formal acceptance dependent on other browsers also implementing it. It’s inherently a implementation lead standardisation process, not necessarily committee led.
Realistically, I think the most likely outcome if there's a formal standard is similar to C: You end up with one compiler that adds extensions to the standard language that are too useful to ignore, and then ~everyone copies them anyway. The most useful extensions are then standardized in the next version of said standard.
Meanwhile, some poor suckers will be working in We Must Follow The Standards hellscapes and will have to stick to the ISO version, and they'll be completely left behind by the community.
*Looks at works best in Chrome sites.*
You sure about that?
Rust has the goal of putting as much smartness between what you type and what the compiler produces, which is a perfectly fine goal, but without a specification for the input semantics it's a pretty wobbly thing, especially for a systems programming language. A lot of bit packing code is simply not writable in rust today without immediately invoking UB, that works for now but might break with every bugfix release.
I don't know about Zig, but as for C, this take is not just wrong, but dangerously wrong. The behavior of C code isn't defined by the underlying assembly, but by an abstract machine model that may or may not match your real machine (narrator: It doesn't). As a result, the compiler can and will ignore the intention of your code when it can prove that your code would invoke UB. Good example: signed integer arithmetic and overflow tests. It is very difficult to write overflow tests for signed integer operations without accidentally invoking UB, and modern compilers will simply remove your overflow tests instead of translating them 1:1 to assembly instructions.
Rust simply lacks that mapping, yet.
However, Rust and C alike have another whole dimension to their semantics, that of Undefined Behavior, which is not reflected in the assembly, and which needs to be taken into account for unsafe code authors (in Rust) / by all programmers (in C). See for example https://www.ralfj.de/blog/2019/07/14/uninit.html for what goes wrong when you think of C as just a macro assembler.
Yes. It's called type safety / type soundness: you cannot cause UB in safe code.
I literally did a PhD on that topic: https://research.ralfj.de/phd/thesis-screen.pdf
But this is no different from C! The way C programs interact with memory is defined in very similar terms to MiniRust's memory interface trait, using "objects" with some rules for which pointers are allowed to interact with which objects.
In both C and Rust, if you want to do pointer tagging or bit packing, you need to consider the language's rules carefully- this does not mean you cannot do it in either case, only that you are stepping close to the boundary of what is supported.
(Maybe that's what you meant, it wasn't entirely clear.)
I'm completely fine with difficult semantics in weird corners of the language. I just want to know them tho :D.
Just today we had this nice example on the front page where initializing a variable differently completely changed the codegen for large parts of the program: https://jpieper.com/2022/08/05/debugging-bare-metal-stm32-fr...
I don't expect to have a good mental model of an optimising compiler in "gotta go fast"-mode, but a mental model of what the memory layout looks like is pretty darn relevant when designing or working with ABIs and binary data representations.
I'm not sure where you got the idea otherwise, but Rust supports the same memory layouts and ABIs as C on a given platform.
You don't get a specific layout by default because that allows for the implementation to improve over time, but that's not really relevant when working with binary data representations, where you simply specify which one to use.
If you only use raw pointers, Rust has significantly less UB that C does.
But if you use references, then yeah we have those aliasing rules and they can be quite restrictive.
You wrote a PHD on it so you might have a better idea of the issues that past me ran into, than what present me can still recall ;)
I'll throw you folks an issue over the fence, should I run into the same problems again, pinky swear ;)
Whats the current state of the art for unsafe code? The guidelines?(https://rust-lang.github.io/unsafe-code-guidelines/)
I found the Rustonomicon, despite it's mythological status, to be quite thin. ^^'
cough https://riscv.org/wp-content/uploads/2018/05/10.45-clifford-...
There are plenty of undefined things in ISAs, writing to some internal ARM registries also springs to mind, but that's a bit of a red herring. Just because code with undefined behaviour doesn't map to some abstract machine model nicely, doesn't imply that code that doesn't invoke UB can't have a simple mapping to some abstract machine semantics.
The VSP hack on Commodore 64 comes to mind.
No there isn't. "Something von neumann-ish" would have behaviour for all inputs - maybe not desirable behaviour, maybe even different behaviour on different processor revisions, but it would have behaviour. The C abstract machine doesn't.
That applies to pretty much all imperative languages that compile directly to assembly/machine code – including Rust.
If your compiler merely translates a source line into a series of assembly mnemonics, function calls, or interpreter gotos, then the interface is the implementation. You can rely on the underlying target language to provide your program with meaning and the only people who have to care are people reimplementing your compiler for compatibility.
The moment you start talking about optimization, then this no longer works. You no longer have a correspondence between source and compiled forms of one program. You have a many-to-many relationship where one source form can be compiled into hundreds of binaries depending on how the compiler is configured, and many source forms may actually optimize to the exact same compiled form. This requires you to provide your own semantics, else compiled programs have no meaning and -O3 becomes shorthand for "make demons fly out my nose".
In the case of C they came up with a series of rules for what-not-to-do that both did not match existing language semantics and also were dangerously incomplete. There are still C programmers who insist that you can free() memory but still touch it for a "little while"[0], or access memory "off the end" of an allocation[1], for example. And ISO C still made the mistake of retaining pointers, which are a confusing mix of value and reference type. They aren't references because you are allowed to cast them to and from integers; and they can't be values because you can use them to modify other values. Because of this tension, we keep discovering new combinations of valid transformations on valid programs that cause miscompiles, and then we have to invent things like pointer provenance to fix them.
As far as I'm concerned, the only difference between Rust and C is that Rust is honest about it's cleverness. C has to pretend to be simple while also out-clevering Rust (or at least, the safe subset of Rust).
[0] Usually in an attempt to emulate automatic memory management. Manual memory management does not work when passing complex structures across an API boundary, and the only options are to either expose custom deallocators (which means no optimizations even when they are sound), tell callers how to deallocate the data (which means no changing the data), or hack the allocator to do what you really want.
[1] It works for malware developers, it should work for me, right?
That seems like an arbitrary criterion. They're values because they behave like values when copying them around etc. You can't even use them to modify other values implicitly - you still need to use * to get an actual dereferenceable/assignable lvalue out of the pointer.
My point is that because pointers act like both values and references, they are neither values nor references. This makes it impossible to soundly reason about them.
There is plenty of PL work (in particular everything considering ML with imperative features, many decades worth of work) where references are values, that works perfectly fine.
Take a look at the languages that rust was influenced by (https://en.wikipedia.org/wiki/Rust_(programming_language)) those aren't languages with straightforward compilation semantics.
There is a reason why rust has a datalog engine build into the compiler (https://github.com/rust-lang/datafrog). Which is imho totally rad and awesome, but really hard to fully form a mental model of without a spec.
The fact that you point to datafrog is illustrative: while it is not actually built into the compiler today, the use case for it is borrow checking, which famously does not impact the language's operational behavior at all! It is purely a compile-time analysis that does not influence code generation.
(For example, consider that mrustc and the GCC frontend are both able to omit borrow checking entirely and yet still produce runnable binaries!)
Of course, the borrow checker is still something that you need to build a mental model for, but by design it is okay for you to get that wrong occasionally, because the result can only ever be "your program still does what you thought, but the analysis proved that in a way you did not expect" or "your program does not compile."
Same applies to the Java and .NET ecosystem, because either you swim on the surface, or you really get to know how the implementations, down to bytecode, JIT, GC and standard libraries work, and now they are full speed ahead with 6 month release schedules.
I've recently tried Zig and switched to it instantly, it's hard to explain but basically Andrew has a very good taste at picking important features and keeping the language complexity very low.
You know how it takes some time to learn borrow checker and macros and generics and traits and all weird rules of what you cannot do and then trait bounds and then it doesn't work exactly like you need, or the crate does not support something and you cannot implement it yourself, etc. etc.
So in Zig I had a hello world on day one, and the first thing I did was encode/decode ANY json messages for tagged union (which unfortunately is not supported in std but it was very easy to do it myself) and it worked! I did this the first day in entirely new language (I was not even doing C/C++ before) and it would probably take few days in rust and I'd probably mess up something and I know it wouldn't work for every case and every crate, because of orphan rule. In Zig it would work for any struct, internal or external. And also Debug, Display, Eq, Partial, all of that works automatically. That's huge!
And the worst thing is that recently, I've started using pointers again, and when I look at the code I don't see anything unsafe in the structure itself, it can be used unsafely but that is also very easy to fix in Zig because you have these explicit allocator and it's so easy to put everything in the same arena transparently, or use SegmentedList with stable pointers.
No it doesn't. "smartness" is a means, not an end or a goal.
> Hear more from Reliability Project Director
https://mobile.twitter.com/rust_foundation/status/1385310806...
Your words, not mine.
I'm fine with a smart compiler. I also think that a high-level language (which Rust is in a sense) also really needs a spec, which MiniRust graciously provides.
This isn't about whether you or anyone else "likes" a smart compiler. This is about not making shit up about the goals that others have.
Maybe I should have said this before, but I'm part of the Rust project. I've been part of it almost a decade now. There has never been any moment or point in time in which I or anyone else I've seen in the project that has "the goal of putting as much smartness between what you type and what the compiler produces." You are just factually wrong.
If anything, it's exactly the opposite. You don't want too much smartness. But you need some of it (to the best of our knowledge) to achieve our actual goals, which, pithily stated, are "a memory safe systems language."
At about the time K&R1 was published, the Ritchie's original C compiler was being replaced by pcc, and `lint` was another early (diagnostics-only) implementation. The K&R1 reference manual includes a few mentions of features that were deprecated by pcc, or new in pcc.
So I think C was more mature than "the compiler is the specification" by the end of the 1970s.
K&R is remarkable for fitting all that into one slim book, and Rust's documentation rightly doesn't try.
Rust's documentation also has some considerable advantages from its modernity. That library documentation gets to link to the actual source code which can answer tricky questions you'd never cover in a book, yet in many cases that source is very readable, its examples are something you can try for yourself with one click, instead of one of the very best human compiled indices of any book it has text search and hyperlinks.
> It wasn't standardized until 1989 - 17 years later!
> I think Rust will do just fine,
Not based on the example you gave - using the behaviour of a single C compiler as a specification for the language implemented by that fairly easy due to how small C is. Rust is very much larger than C.
> and the compiler error message is (IMO) the best of any language, making adhoc exploration possible.
> Can you imagine trying to explore C from ad-hoc alone based on its compiler errors?!
Not much different from Rust: Gcc and clang C compiler errors are exceptionally good, and I miss them terribly when programming in some other language (like C++, or Javascript).
I wish i still had my late 80s copy.
Not mainstream language 'fine', but fine.
Type "Rust" into a job search engine and see what comes up, it's mostly blockchainy stuff. Or a few shops that are "looking into" doing Rust for new projects (which is encouraging, but hard to say how that will shake out).
I personally won't touch anything "crypto", so I feel somewhat lucky to be getting paid to work in Rust right now for something that isn't. But I fully expect that once this gig finishes, it'll probably be back to C++ for me.
Rust usage at these companies isn't yet 'large' in the sense of KLoC or anything, especially compared to existing legacy codebases, but it's getting put in at some really critical infrastructure levels that are unlikely to do anything but grow imo.
That's kind of the thing about rust though. That's where it's got the most power to improve things, so its introduction to a workspace is kind of subtle.
Is there any public source for Apple using Rust?
And yeah those are exactly the domains Rust is designed for, and even if it doesn't mean huge codebases, that kind of usage at a whole bunch of huge companies hopefully means Rust is more likely to survive beyond the initial hype. :)
But a) it's Apple, a stressy, workaholic company to work for [at least that's how it seemed when my wife worked there] and b) I'd never convince the wife and kids to move right now.
It's a surprising location and the story behind it existing is fun but also not mine to tell, but if you applied and interviewed you'd almost certainly get to hear it. The stuff that team does is much more important than its location would suggest and the software they build was mostly originally in C (not c++), which is also kind of surprising for the services side of apple. We were involved in porting swift to Linux in the hopes it would meet our needs but that didn't work out for various silly reasons.
At any rate it's not the only part of apple using rust, but it is probably the earliest adopter and also the most mission critical user there.
> But a) it's Apple, a stressy, workaholic company to work for [at least that's how it seemed when my wife worked there] and b) I'd never convince the wife and kids to move right now.
Fwiw this team isn't really any of those things, it's very unusual in a lot of ways. Very good work life balance, especially now that the office isn't an hour from a real city.
I didn't leave on bad terms at all, I just needed a change of scenery and the one big downside to that office is you can't really go to any other teams because Apple is so dead set against remote or distributed teams without a shared office space, so you're just kinda stuck. Also I ran out my patience for not being able to work on open source unrelated to my job (though at least it got easier to work on open source related to my job over the time I did work there).
I would characterize the amount of Rust being done at Google as ... insignifant.
That could change, but apart from Fuchsia and maybe some kernel driver work I struggle to picture where it would fit in the language pantheon there.
I am in contact with some people in the Android team at Google that are also using Rust (and that want to use Miri :D ).
But I'm sure it's growing. But it won't be in the core Google3-on-Borg parts of Google, which are on the whole in either C++ or (increasingly) Go. Switching to a standardized garbage collected Google supported language makes sense in that domain. The "Carbon" stuff recently announced sounds interesting as well.
The long term looks good for Rust. But actual production code is still thin on the ground still everywhere.
FWIW, to the best of my memory as of a year ago I counted ~1.1 MLoC of Rust code in our open source tree, with an additional ~1.3 MLoC in things we've vendored from crates.io.
There was a bit more C++ when I did the analysis, but it was the same order of magnitude.
Other than that... Fuchsia was the thing that was rewriting all my work from scratch and killing everything I worked on (Chromecast on Home Hub, etc), so... :-)
Those are just the loudest people. Web3 companies depend on hype so they shout all over the web about anything they can, including how they're using the hot new language for blah blah blah and you can come and write it with us! (As long as you're willing to be paid in worthless tokens.)
In normal companies there is plenty of Rust use but you don't hear about it as much because it tends to be initiated internally by people who weren't hired as Rust programmers, and it's used for new projects that take a long time to get to the point of needing new employees. I know in my company it was at least 2 years between when we started using Rust and when we wrote "Rust" on a job advert.
Do a search on Upwork for Rust gigs. Almost all crypto.
LinkedIn jobs, very similar. Putting "Rust" in my profile there has also led to a lot of recruiters pinging me about web3-ish jobs that I have no interest in.
The Rust PL group I'm on, also on LinkedIn, has a steady stream of 'smart contract' etc discussions. And people advertising themselves for crypto jobs. Or crypto jobs advertising looking for people.
That's not to say there isn't plenty of other Rust stuff out there, but it seems like this is where a lot of the use is right now and I really hope the language as a whole doesn't get typecast this way.
My overall thought is that programming language popularity is a lagging indicator; people might be hiring Java engineers even if they wish they didn't write their codebase in Java. Changing that is risky; getting one more employee can stop the bleeding. Meanwhile newly-formed companies use the newest tools, and thus among newly-formed companies, you're going to see many more requests for engineers with experience with the newer tools. (It's one of those secret advantages for startups; your competitor might be slowed down by all the legacy and thus resources given to your new startup might get more value than if given to the incumbent.)
by dictionary definition, rust is not mainstream. it's not normal (common, typical, customary) for an engineer to interact with rust
It's only 3 times less popular than C. As I said, very weird definition of mainstream.
It will come. It's not there yet.
I still argue it's not the language best suited for what a lot of people are trying to use it for. It has a niche, one that I like, but I fail to understand why it's being deployed for web services type applications. IMHO it's not a good fit.
There are plenty of reasons why it's a good fit for web services. It's fast, has great tooling, a great library ecosystem, and perhaps most importantly the strong type system and borrow checker mean you write far fewer bugs than in most other languages (C++, Java, Typescript, Go etc; maybe not Haskell).
The only reason I wouldn't use it in web services is because the whole ecosystem seems to have decided that everything has to be async despite that adding a ton of complexity and almost nobody needing it.
But none of this is unique to Rust. And I'm not convinced on the WASM stuff at all for writing web front ends, though I'm enamoured of it as a generic backend managed VM, but that's another story.
And I think adopting Rust in many shops with a mix of developer seniority levels will be a productivity killer. I think most places doing this kind of thing would probably be better off with TypeScript, or Kotlin. Or if they want to go more functional and exotic, F#, Scala, OCaml, or Erlang/Elixir, etc.
But for places where we've been doing C++ for years and setting off footguns all over the place? Rust is great! Love it.
It's pretty tricky to do that in an untyped language. But TypeScript could have this, or maybe even already does?
AFAIK TypeScript doesn't have ML-style pattern matching, but its type system is fairly expressive and you can do something "like" it https://dev.to/gvergnaud/bringing-pattern-matching-to-typesc...
If I just say "algebraic data types" or "sum types" I fear many people won't know what I mean...
If you mean dynamically typed, then Elixir has it:
https://elixir-lang.org/getting-started/case-cond-and-if.htm...
What I was referring to is things like `match x { Some(y) => ..., None => ... }`, where there is data that is available only in some variants of the type (like `y`). Without a type, even naming these variants becomes an interesting problem. (Not unsolvable, mind you. But not obvious either, and you lose some of the benefits that algebraic data types have in typed languages.)
How is that not mainstream by even your own definition? It’s consistently a part of the current zeitgeist.
But what is interesting I think is that if miniRust is successful (or even if it isn't) zig can learn lessons on how to do borrow checking on zir: even if it isn't part of the official language release it could be a useful tool for ci/linting/touchup, especially if it becomes an "anointed" tool
I'm much more interested in specific details of how to permit mutable aliasing while simultaneously avoiding use-after-free. From what I've seen, the only compelling answer is a garbage collector.
So far no language not designed for it has demonstrated that adopting a borrow-checker to interesting real-world use-cases is possible. The C++ project (C++ Core Guidelines it is called IIRC) eventually gave up on soundness. Zig isn't there yet, I am very curious how they will tackle this.
So, I'm not going to use the word "impossible", but I think this is a hard problem and many people are underestimating how hard it is. It is strictly harder than what Rust is attempting, and see all the tricks they had to pull!
Put differently, if you are right, it should be easy to prove pcwalton wrong by just doing it. :)
The interesting question is whether there exists some system fundamentally/significantly simpler than the rust approach which still gives you comparable freedom and safety (without excessive runtime overhead).
As far as I know that is unsolved and a matter for research, not some obvious solution that the rust team totally missed when designing their language.
If your goal is lower than total memory safety, then obviously you'll have more options, but when it comes to memory models, you have to be quite careful in where you allow violations, since they tend to infect everything quickly if you don't find exactly the right way to encapsulate them, so having a "half-safe" language often just means you have an unsafe language. Part of the reason unsafe rust is so painful is they're adding exactly that sort of hole in the memory model, and suddenly anyone exposed to it can't deal with the "simple" memory model of rust, they have to deal with all the gory details _creating_ that simple model. Any language adding larger holes than unsafe will need more people to be aware of the full complexity of modern compilation more of the time, which doesn't seem like a win. Rust's unsafe works because it's almost never used. If unsafe was half the codebase, rust would be a failed language.
Another solution is to absolutely refuse to take advantage of your safer memory model in the compiler - so minimal optimizations, just compile literally and throw barriers all over the code, then anyone using a downgraded model doesn't need to know the additional rules they would otherwise need to follow to interact with the full model. That sounds pretty awful though, probably more expensive than just using a garbage collector.
Is there a better solution? Sure, probably, but I don't think humanity has discovered it yet.
The real problem is pretty much any model that doesn't shoot for complete memory safety (or a unsafety that is strongly discouraged and well encapsulated) is going to end up containing all the complexity that comes from unsafety, rust justifies its complexity by all the complexity it also removed (undefined behaviour is basically not a thing in Rust), if you half the complexity compared to rust but retain half the undefined behaviour complexity of C++, I think you'll have just grafted the worst of both together. I'm unconvinced there's a sweet spot in the middle, more like a mountain of extreme complexity (ala rust unsafe) between 2 valleys of relative simplicity (C-like, rust-like).
Is there any way to have Zig output a flat binary? I am looking for a higher level FlatAssembler. [0]
It's not "useful" in the sense of being something that you'd actually run.
This is a start on the lowest layer of the description, describing the "abstract machine" that runs Rust programs; it assumes that all borrow-checking and type-checking has already been done, generics have been expanded, and so on.
The other two big pieces that will need doing are properly describing the type system, and properly describing how the high-level features that aren't strictly necessary can be "lowered" into simpler forms.
In the last year there's been some real work on the first of those under the name "A MIR Formality": https://nikomatsakis.github.io/a-mir-formality/docs/intro
Maybe the existing Rust Reference will turn into the second.
What if we could actually run it (stay with me for a second) with a well known interpreter such as python3 or v8?
That is the direction py2many has taken. Trying to define a small subset of python + missing rust features (pattern matching as an expression), interfacing with formal verification methods such as z3 and then finally building a transpiler to transpile this language to rust.
- Testing the spec itself - Ensuring that a more production-grade interpreter like Miri has the same semantics as the spec it claims to implement
It's not intended that unsafe Rust authors would ever run the MiniRust interpreter; if they want something like that, they should run Miri instead. They might still read the MiniRust interpreter sources to figure out what the heck Miri is doing.
If you are writing a Rust compiler or a Rust verification tool, then a spec like MiniRust is a key part of telling you whether you got it right.
If you are not doing either of these things then you probably don't have a use-case for MiniRust. :)
lol, I didn't even realize the wrong associations I would create here. Glad you liked it anyway. :)
I guess my choice of name was not great. I also considered pseudo-MIR but that didn't really capture it, either. And "formally specified fragment of Rust" isn't very catchy.^^ "CoreRust", maybe it should have been "CoreRust"...
I'd personally pretty much always expect "mini" or "r" (as in "rperl", a restricted subset of Perl with C++ connections) versions of a language to be restricted subsets for some purpose (rperl's is to give away flexibility for performance while maintaining a good portion of the original language).
I've seen an "e" or "emb" prefix or a "small", "tiny", "micro" or "µ" (or "u") prefix to mean a small toolchain version several places, like SmallC or uclibc or Mikroe's mikroC (https://www.mikroe.com/mikroc). It wouldn't surprise me to see a "nano" version of a language tool either. Sometimes these are subsets as well, but to fit the size constraints of the target rather than for constraining the input for its own sake.
Programming in NQP is troublesome, but it is fast and it is self-hosting.
The two projects are related, but have different objectives (mir-formality includes traits and borrow checking, while MiniRust focuses on operational semantics).
Not only that the spec will be clearer but you can get more contributors on the core part of Rust.
Cool bear has breached containment.
I do have to admit I love the recent-ish trend of including dialogues in blog posts. Plato would be proud.
My only worry regarding Cool Bear is the licensing of the two pieces of artwork: I bought them a while ago on some stock image website but the terms weren't super clear, so I might need to revisit that. Since then I've commissioned drawings of bear and myself (5 variants each) but I haven't had a chance to use those yet — they're not monochrome, which makes dark mode awkward.
[1]: Living with the Rust trademark [LWN.net] https://lwn.net/Articles/901816/