Rust const generics MVP hits beta
blog.rust-lang.org
blog.rust-lang.org
But also, beyond that, this is pretty much the last major feature that I've wanted in Rust. I've got some long-form writing in my head about this, but previously, I would have cast it as two or 2.5 features:
* const generics
* GATs
* specialization (this is the half)
However, when I've started to think about explaining these sorts of things, GATs (and to some degree specialization) feel much more to me like a removal of a restriction on combinations of features, more than "new features" strictly speaking. I think the line between these two perspectives is fascinating. On some level, you can even cast const generics in this way too: what it's really doing is making arrays a first-class feature of the language. I think that's a bigger stretch than GATs though, so I am not fully sure I'd make that argument. (The minimum feature we're stabilizing now basically does only this, but we do plan on going farther, so it feels true on a technicality now but not later.)
Regardless, it's been nice to see how much we've slowed down in adding major things. November of 2019 was the last time we had a feature this large hit stable. 18 months between huge things feels much more like the cadence of more mature languages that have been around a lot longer than Rust.
There is still a lot of work to do removing restrictions on existing features, especially const fn and const generics. But Rust is really starting to feel "done" to me, personally.
struct Person<F> {
pub name: F<String>,
pub age: F<u32>,
}
[0]: https://reasonablypolymorphic.com/blog/higher-kinded-data/
All that said, I'm really excited to have const generics land! Props to all the amazing work by withoutboats and the entire rust team.
https://github.com/tim-group/higher-kinded-lifecycle/blob/ma...
(in this code, "idea" is a domain concept from the firm i worked for at the time - basically a recommendation to buy a stock, which is 'opened' on a certain date, and 'closed' when it no longer seems like a good recommendation)
My collegues didn't like it, and stuck to using separate types for objects in different stages of the lifecycle!
type Person = {
name: string;
age: number;
}
type SomeFieldsMissing = Partial<Person>;
type AllFieldsRequired = Required<Person>; // No real change in this case
type Partial<T> = { [P in keyof T]?: T[P] | undefined; }
type Required<T> = { [P in keyof T]-?: T[P]; }
struct Person<F> {
pub name: F<String>,
pub age: Option<u32>,
}
(Also, it shouldn't be Box; you'd want a bog-standard generic newtype for this, not an extra heap allocation.)
If one caller passes our type narrower with {age:F<u32>} then the return value of the function will be narrowed to that type, but a broader criteria can still be specified for the generic instance.
type RequiredKeys<O, K extends keyof O> = Omit<O, K> & Required<Pick<O, K>>
type Person = {
name?: string
age?: number
}
type RequiredName = RequiredKeys<Person, 'name'>
type OptionalKeys<O, K extends keyof O> = Omit<O, K> & Partial<Pick<O, K>>Edit, for that matter, how do you deal with a `Person f` that includes a `f (Maybe Something)` without confusing "the user didn't specify a `Maybe Something`" with "the user specified a `Maybe Something` with value `Nothing`" (or vice versa)?
type Maybe<T> = {hasValue: false} | {hasValue: true, value: T}
type Shoehorn<T> = { [P in keyof T]-?: Maybe<Maybe<P>> }`Compose Maybe Maybe T` (equivalently `Maybe (Maybe T))`), has N + 2 values (where N in the probably-infinite number of values T has):
. N values consisting of a T
. 2 different values that do not include a T
It's also equivalent (via 1 + (1 + N) = (1 + 1) + N) to `Either Bool T`.
And yes, `Person f` contains values of type `f Whatever`.
> it either is undefined or isn't.
Values are never undefined. They might be defined as a object that's called "undefined", but that is a specific, defined, value. And, apropos of the above, it's a single value, out of the two distinct non-T values being represented. Which is part of why the example at https://news.ycombinator.com/item?id=26277598 looks horrible - it seems like it would confuse the two different non-T values with each other if `Person f` includes a `f (Maybe T)`.
Edit: actually, clarified the previous comment a bit, too.
I have no clue what "N + 2 values (where N in the probably-infinite number of values T has): N values consisting of a T" means and I've done fairly well for myself.
Why does anyone need this to write and deploy CRUD apps and put forms + buttons on stuff?
I kid, but I admit there is a valid division between coding to explore the consequences logical frameworks and coding to extract money from corporations while providing them a way to keep their orders coming in and accounted for. I'll gladly spend time on both sides of the spectrum, depending on how many drugs are in my system and how much food is in my belly :)
> Values are never undefined.
I am referring to the JS concept of the singleton `undefined`, not an abstract idea of definedness.
Overall I see where you're coming from, but scrambling Haskell and TypeScript syntax is severely impacting legibility.
End of the day Haskell and TS have very different use cases, one is a research language exploring the reaches of type theory, the other is type system bolted onto a dynamic language doing the best it can. Sure there are some concepts that it can't express, but that's entirely missing the point.
https://rustyyato.github.io/type/system,type/families/2021/0...
What's a practical use case for it? It's cool in theory, but I struggle to come up with something that doesn't use at most 2 implementations (just copy the struct then), or some API where the types don't mix and can be generated via a macro instead.
I've been living on savings in self-sabbatical mode while catching-up on life, projects, and reading while keeping current with Rust.
I'm soo itching to jump back into workaholism mode.
https://news.ycombinator.com/item?id=23975445 is probably the most accessible thing we've talked about publicly with what we're doing. (This talk was what made me decide to apply.) Still early enough the focus is on building the product rather than talking about it to broad audiences like HN.
Excessive, horn-tooting quasi- resume/cover letter in an HN comment (maybe I should've compressed and uuencoded?).
Home: Austin downtown, via much of NorCal { Chico, Sacramento, and Bay Area }, from San Jose.
Experience with embedded systems on many architectures and different types of RT deadlines, SMT electronics (PCB design and fabrication, stuffing, and repair), Rust (I sure hope so!), C/C++ (autodidacted at 15), LLVM, several assembly languages (ARM, X86, MIPS), microcontrollers, large-scale industrial firmware development (automotive, mining, & agriculture).
I periodically throw 3D printing and Arduino or ESP32 at personal use-cases. And trinocular microscope, Siglent scope, and knock-off compatible JBC solder station. The Fluke 289 DMM is real though.
Can do everything from sales engineering, on-prem/remote custom client integration, customer training, feature development, tools support, customer bug forensics, subgroup lead SWE, systems architecture/engineering, customer solutions development. Make the office more funner now and then (especially internal funny 404 pages and easter eggs, tape your chair up, orbeez explosion, get your wife/SO and kids in on a gaslighting long-con). 5¢ charge per instance. Wooden nickels not accepted. No refunds. :)
Did one or two minor humblebragging things:
- Ported a ridiculous nuclear reactor simulator in Fortran from UNIX to Windows.
- IBM Almaden offered me a dark matter research assistant job when I was 15, but my parents weren't supportive and it would've been illegal. :Y U No moon meme here:
- Wrote a Java-- to native MIPS (non-JIT, compile-time) compiler from scratch with symmetrical implementations in Java and C++.
- Refactored the heck out of a UDP 900 MHz packet radio firmware C++ codebase and added telemetry logging with error tolerance of Flash EEPROM wear beyond mfgr specs.
- Restoring & modding a VW camper and getting into paramotoring (parachute wing and motor on your back).
Please apply! We are hiring.
You certainly aren't replying to you applicants, though.
EDIT: I decided to take a look, and it seems like you applied before I joined. My apologies, it must have been a mistake.
Well that mistake seems to keep repeating itself then.
Oxide has been silent even after subsequent contact, and even after this issue has already been raised internally once.
Yeah, I've been pining for highly-integrated, hyperscale datacenter metal. Only Fortune 50 corporations and partakers of OEMs/ODMs like Quanta QCT / Dell DCS deliver somewhat okay customized gear, but not a completely-integrated, managed fleet. I remember eGenera BladeFrame, but it's still nothing like a hyperscale, intermodal-container LEGO datacenter solution.
Oh hey from I-35 and 5th!
Here's your "I probably survived the 2021 ATX Icepocalypse" tie-dyed hoodie with black light effects.
Tangent: What's your take on the Rust HALs? (eg `stm32l4xx-hal`) etc? I'm curious what the take is of someone outside the Github and Matrix communities. They strike me as... remarkably underdeveloped, but have big potential.
In today's Rust, data structures often need to be a fixed size, or dynamically allocate. This will let you write a data structure that has a variable size, but set at compile time. I knocked out a quick ringbuffer recently; I picked a size that seemed fine, but that size is fixed at the moment. If I had used const generics, it could have been more flexible here, I could have written it more easily for any size, where the size is written at compile time. This is not my work code, but back when I was working on a hobby x86 kernel, I did this for a VGA buffer: https://github.com/intermezzOS/kernel/blob/master/vga/src/li... it's generic over "something that can be converted to a slice," so that it uses a slice in production, but a Vec in tests. This also leads to runtime checks https://github.com/intermezzOS/kernel/blob/master/vga/src/li... to make sure that the slice is the right size. This would be much better written with const generics today. The slice trick is neat, but awkward. (This code is also awkward because it grew an internal "frame buffer," so it kinda has both going on. This example needs const generics less since VGA has one single size at compile time ever, but sometimes, you need something where this isn't the case, and this was the first example of code I personally wrote that springs to mind.)
> What's your take on the Rust HALs?
I haven't used them enough. We use the layer below them, for example, the stm32f4 crate rather than the stm32f4xx-hal crate. That doesn't mean they're bad, I just literally have not used them enough to form an opinion. The namesake of this space is diversity, and so different people make different tradeoffs; we don't really need what those crates are offering right now.
pub enum Expr {
Int,
Str
}
pub enum Expr2 : Expr{
Bool,
}
fn check(??)-> Expr.Int struct Person {
id:i32
}
struct Customer: Person {
}
let input: Option<usize> = Some(1);
if let Some(value) != input {
return 0;
}
// use value; such that value: usize = 1 let input: Option<usize> = Some(1);
let value = match input {
Some(x) => x,
_ => return 0,
};
// use value; such that value: usize = 1The point of a 'guard' clause is to visually distinguish between the "expected" and "unexpected" case. When I see a guard statement:
let input: Option<usize> = Some(1);
guard let Some(value) = input else {
return 0;
}
// use value; such that value: usize = 1
guard let x = ... else {
// must not fall through, so a return, break, throw, non-returning function call, etc. is required
}
// do something with x let x = match foo {
Some(x) => ...,
None => return // or break, etc.
};
// do something with x
// want to do this
if not let Some("pattern") = val {
doSomething();
}
// can instead do
if !matches!(val, Some("pattern")) {
doSomething();
} if !matches!(val, Some(let x))However (and I’d love your thoughts about this) the one feature I feel has not gotten the love it deserves is DST types and safely creating them. I was writing a zero-alloc no_std h264 and mpegts parser and the hoops I had to jump through to represent extant mpeg structures as valid rust data types was insane and I ran into a lot of edge cases with transmute that made me scratch my head. In the end, I wrote my own macro for creating (in-place new-ing) DST types that transmuted fat pointers (whose representation is not guaranteed but unlikely to change anytime soon) to get things kind of working but the ergonomics and safety were non-existent.
I feel like if rust doesn’t pick up the baton for innovating on DSTs it would be a huge shame since from a security and memory safety perspective they cause the lion’s share of overflow and out of bounds accesses in parsers and other high-risk libraries that silently digest all data thrown their way for media previews, thumbnail generation, etc etc.
What irked me is that a lot of people consider the checkbox for DST support to be ticked when there’s really only the bare minimum for interesting with types containing single DST elements that are already - somehow - created, typically by the standard library itself.
(Again from an embedded perspective: I’m really happy with async/await for embedded development; it really makes reasoning about control flow in an embedded context much more sane!)
For example, when writing a `GEMM` (general matrix multiplication) routine, the core building block is a so called kernel, which gets moved over the matrices like a stencil. The size of the kernel however depends on a) the numerical precision (float, double), b) the available SIMD intrinsics, and c) the architecture the code is executed on (haswell, skylake or zen have different latency and throughput for different intrinsics).
Right now it's surprisingly painful to write an optimal kernel and buffer, which is different for every architecture and which you need to kind of hardcode. With const generics this should become much easier.
I am a huge fan of nalgebra and having const generics would make the API and implementation much nicer.
Hopefully the robotics industry can move towards implementing stuff in Rust. Having low level robotics algorithms like state estimation and SLAM in Rust makes perfect sense.
A lot more.
A really whole lot more.
"Pulling teeth" is how I'd describe ndarray currently.
struct DiscardPastN(uint len)
{
int[len] buffer;
uint state = 0;
void put(int x)
{
buffer[++state % len] = x;
//mwuhahahaha
static if(len == 4)
asm {
ud2;
}
}
int[len] get() const
{
return buffer;
}
}
For example you can't use associated constant as generic arguments, preventing constructions like this:
trait HashFunction {
const OUTPUT_SIZE: usize;
fn hash(input: &[u8]) -> [u8; Self::OUTPUT_SIZE];
} pub trait HashFunction<const OUTPUT_SIZE: usize> {
fn hash(data: &[u8]) -> [u8; OUTPUT_SIZE];
}
Also for the Rust crowd, is it possible to do something like this?
pub trait HashFunction<const OUTPUT_SIZE: usize> {
// I want Output to always be [u8; OUTPUT_SIZE], but using associated type means
// whatever implements HashFunction can redefine Output.
type Output;
fn hash(data: &[u8]) -> Output;
}Serde proper needs something like const_evaluatable_checked before it can offer large array support.
There were workarounds for some common scenarios already, but it's just so much more simple and convenient to be able to write `fn foo<const N: u32>()`.
I'll finally be able to simplify a significant portion of my code.
I use rust for a (to be released -- comment with your github if you want early access) Bitcoin Smart Contract embedded domain specific language (edsl) that operates sort of like a circuit meta-programming language. Having const generics will drastically simplify my code and enable greater "structural type safety" (checked at the rust level rather than as a part of my edsl).
They're not wrong when they say that this is the most highly anticipated feature :)
Very needed and cool
This seems to make it easier to do stuff without allocating heap memory.
How large is the stack? Can I do complex things with only the stack?
And yes, you can do many things using the stack; it's just not as flexible.
But const generics are most useful for increasing the efficiency of certain code while keeping it reusable.
e.g.
struct SmallString(size_t N) { /* etc. */ }Consider the design effort behind parametric memory allocators. That seems like a pretty cutting-edge problem. And yet I bet the Rust folks knew that they would want/need to do this way before they started doing that work in earnest, because people from C++ seem to want the same thing (if they don’t have it already?).
I idly wonder if one could, if one was in a similar position as Rust was some years ago, just go ahead and design a full-on unapologetic type-level programming language from the start. Because you know that your type-level terms will be worthy of the moniker “language” eventually (and it might not be a compliment as such).
Just a nice, high-level language that doesn’t bother with the “bare metal” concepts that Rust the value-level language has to deal with. (Can it even be done? Don’t ask the peanut gallery about that.)
Either that or you accidentally build an emergent language that Gankro can write an article about one day titled, I don’t know, Shitting Your Pants With Higher-Order Unsafe Unwind Type-Level Allocator Escape-Suppressing Storm Cellars.
[1] In this case: you have more use for compile-time integers than something more general like being able to describe that two nat-indexed lists are of the same length, like you can in Idris.