Why I rewrote my Rust keyboard firmware in Zig: consistency, mastery, and fun
kevinlynagh.com
kevinlynagh.com
For a generic led driver, it should not use these types, but instead the trait types from the embedded_hal crate, such as "OutputPin" that is implemented by the different chip-specific HALs. There is an example of a generic led driver that uses these traits at [1].
In general I recommend everyone who wants to try out Rust on embedded to read the embedded rust book, because it clarifies a lot of the reasons and advantages of its approach.
[0] https://docs.rust-embedded.org/book/static-guarantees/typest...
[1] https://github.com/drogue-iot/drogue-device/blob/main/rt/src...
However, that is not what is happening here. P0 and P1 are distinct types because they are distinct hardware registers. I think it's great that they're modeled as distinct types; the problem is simply that Rust makes it difficult to conceptually iterate over distinct types (regardless if such iteration occurs at runtime via a loop or at compile-time via an unrolled loop, as per Zig's `inline for`).
An aside about "type state programming": Microcontrollers have a lot of functionality packed into the pins (see the STM32 "Alternate Function" datasheet tables). Trying to model all of that using ownership of zero-sized generic types would strike me as a "when all you have is a hammer"-type situation.
If a single pin switches between, for example, high-impedance, gpio low, and PWM output depending on what mode your firmware is in, I suspect it'd be a nightmare to pass owned types around Rust functions --- one would have a much easier time (and more likely to be correct) if they checked their design using something like TLA+ / Alloy or implemented the firmware using an explicit statecharts runtime like Quantum Leap's QP framework https://www.state-machine.com/.
struct P0{}
impl P0{
fn write(self:&Self,pin:usize){
std::println!("Writing port P0 on pin {}",pin);
}
}
struct P1{}
impl P1{
fn write(self:&Self,pin:usize){
std::println!("Writing port P1 on pin {}",pin);
}
}
macro_rules! for_each_port_pin{
($port:ident,$pin:ident,$b:block,
$(($e1:expr,$e2:expr)),*) =>{
$(
let $port = $e1;
let $pin = $e2;
$b
);*
}
}
fn main(){
let p0 = P0{};
let p1 = P1{};
for_each_port_pin!(port,pin,
{port.write(pin);},
(&p0,10usize),(&p1,7usize)
);
}
enum MyPin { P0, P1 }
Edit: feel free to ignore, read your answer somewhere else about this
You would then have to pattern match when you read the value but I don't see a reason to reach for macros or anything more complicated.
That said, really enjoyed the read (and I'll definitely try zig at some point, if only for the speed / compile experience), even if my experience with Rust didn't match yours; my background is a bit different though, I worked with C++ and Haskell in the past, which definitely made rust feel almost natural. Overall I'd say that the compiler helps me not to keep a lot of rust syntax in my mind and just try things until it works
I second this. The idea of checking that a pin is "only used in one place" doesn't really jive with how I think about microcontroller programming. It's very common for one pin to be used for multiple distinct purposes at different times.
There's also a lot of different ways of conceptually slicing pin states. For example, if you are charlieplexing LEDs than you'll switch pins between 'input' (high impedance) and 'output' modes, but at a higher level the pin is serving a single function.
The borrow checker is not checking that the pin is used in "only one place", it is checking that you don't use the same pin for two different purposes at the same time.
It make sure that you configure your pin as output pin before using it as an output pin, and that you reconfigure it to input pin when using it as such.
(And there are some escape hatch to use when the type system is not sufficient to express that different code paths are disjoint, like RefCell, with runtime check, or unsafe)
That seems sensible, but also not particularly valuable. A lot of the time it makes sense both to 'read' and 'write' from a pin (e.g. if it's open-drain with a pullup).
Borrow checker tracks who is using what over time. The can prevent concurrency and uncoordinated mutation, use after free type problems.
Type system checks how it is being used.
Both are tools and can used to help ensure a correct program. It really comes down to how these _tools_ are used to help the programmer and the team deconstruct and manage a system to solve a problem.
I think petgraph [1] is an excellent example of relaxing some of the constraints imposed by the tools (borrow checker, type system) to make a system that was easier to use and extend. These things are much more continuous than we realize, it isn't all or nothing.
In a lot of ways, I wish Rust's complexity was a little more gradual, or that we knew how to use it in a gradual way. Use of Rust features encourages C++ levels of complexity. Use of Zig features encourages C-ish levels of complexity.
Zig is to C
as
Rust is to C++
I also think the author had a much better model of the system and the hardware and what they wanted to accomplish during the rewrite and could better project the problem domain and the language together.
Learning Rust and the problem domain at the same time is extremely difficult and in a way leads to a perversion of both.
What do you think about modeling the hardware as a "Resource" register, port, memory, etc. Then modeling a type over a collection of resources.
The question that I would ask myself when trying to use Rust and the features it has out of the box is, "How much fine grain rigor do I want Rust to model for me?" For the keyboard scanning code, in asm or C, one might just have a function `get_keyboard_state(*keyboard_buffer)` but this exposes a sampling problem and would require the programmer to determine state transitions. So maybe a channel or an iterator would be better. Then we might nee to run it in an ISR, the hardware it uses might be multiplexed with other parts of the system, etc.
Every Rust feature needs to be weighed, or rather, given a complexity budget, every Rust feature needs to be purchased.
Zig is awesome BTW, but it doesn't make the same correctness guarantees that Rust can.
[1] petgraph, https://docs.rs/crate/petgraph/0.5.1
This is a very imprecise statement. Do you mean tracks at "compile time" or at "run time"?
A more accurate statement would be -- the borrow checker enforces a specific set of rules and constraints at _compile time_ only. But this set of constraints guarantees memory safety at run time (with the exception of unsafe code). In fact, Rust's runtime is minimal -- it handles panics but has no GC or other bells and whistles. The fancy things are in libraries.
Anecdotal, but as someone who works in this space I haven't found this to be the case. In my experience, any particular pin is wired up for a specific purpose, and so the firmware usually just sets it to that mode as appropriate. Generally if it's found that the needed peripherals couldn't be multiplexed to pins without conflicts, it's time to move up to a package with more pins brought out.
I'm currently working on a relatively involved firmware for ATSAMD21 in Rust, and have mostly enjoyed the experience. While some of the language concepts have taken me a while to get comfortable with, and we're still figuring out parts of the ecosystem, it's quite usable and the tooling is a huge improvement over anything else I've seen.
I'm with you, but ...
I find that most SDKs invariably grow a C macro system for "Configure this pin/register/whatever and yell if somebody tries to reconfigure it".
The fact that Rust is baking this in up front is not unwarranted.
> Microcontrollers have a lot of functionality packed into the pins (see the STM32 "Alternate Function" datasheet tables). Trying to model all of that using ownership of zero-sized generic types would strike me as a "when all you have is a hammer"-type situation.
The whole idea of utilizing TLA+ for a system level check really does seem like something that would be awesome, even if it's unclear how much effort it'd require to instrument an entire project with TLA+.
> the problem is simply that Rust makes it difficult to conceptually iterate over distinct types (regardless if such iteration occurs at runtime via a loop or at compile-time via an unrolled loop, as per Zig's `inline for`).
Rust just brings a lot of incidental complexity along and still makes some things really difficult. Perhaps it's better in the long run but it's just harder to work with.
Similarly, I wanted a simpler language than Rust and started using Nim last summer for embedded projects. Primarily since it compiles to C which let's me target ESP32's and existing RTOS'es without rewriting everything or trying to get LLVM based tools to work with GCC ones. However, it also embraces `lazy` compilation approach to code and it's standard library.
I wanted to try your example in Nim. Here's roughly how your example would look in Nim (it appears to duck type fine as well):
var
# normally just import these from the C headers as-is
# but this lets us run it
p0* : RefPort0 = addr port0
p1* : RefPort1 = addr port1
var rows = (
( port: p1, pin: 0 ),
( port: p1, pin: 1 ),
( port: p1, pin: 2 ),
( port: p1, pin: 4 ),
)
var cols = (
(port: p0, pin: 13 ),
(port: p1, pin: 15 ),
...
(port: p0, pin: 2 )
)
proc initKeyboardGPIO() =
rows[0].port.pin[rows[0].pin].dir = output
for item in rows.fields:
item.port.pin[item.pin].dir = output
I've toyed with the thought of adding TLA+ hooks into Nim similar to Dr Nim (https://nim-lang.github.io/Nim/drnim.html) using the effect system. Not sure if Zig has an effect system for a similar setup.
You could do those things in C++ with template games and in Rust with macros. But Zig lets you have immense expressivity with a simple, small and easy-to-learn language.
This is what made me love Lua for embedded programming. The more inherent complexity (or "exposed complexity" might be a better phrase) in the system, the less inherent complexity you want in the language.
const fn is (or seems to me to be) exactly what comptime is though. The difference is that rust's const syntax is still slowly allowing more things to be executed at compile time. Like for now, it still can't do any heap allocation.
Given the code in the blog post, yes. Here's a possible solution:
pub fn initKeyboardGPIO() void {
comptime checkPinsAreUnique(10, rows);
comptime checkPinsAreUnique(100, cols);
...
}
fn checkPinsAreUnique(max_pin: usize, elems: anytype) void {
var seen = [1]bool{false} ** (max_pin + 1);
inline for (elems) |x| {
if (x.pin > max_pin) {
@compileError("Found pin value higher than max pin");
}
if (seen[x.pin]) {
@compileError("Found duplicate pin!");
}
seen[x.pin] = true;
}
}
There's also other ways of approaching the implementation depending on the required level of dynamicism, I just hacked together the quickest solution I could think of.
In Zig I think you can get 99% of the benefit by setting up a framework where you allocate a resource (pin, ppi channel, etc) through a function call. We use this for a testing framework which gives you run-time errors. But with Zig you could probably write this in a way that gives you compile-time errors for statically allocated resources. That should give you a system that works in both compile and run time.
Yeah, you can't totally guarantee that a pin isn't allocated, since a programmer can use the pin without calling the resource allocation function. But I feel like that takes you from 99.99% safe to 99.9999% .. worth in in a few obscure applications, but not in most.
It's not like I've ever seen any issues from allocating a pin twice in embedded programming. On nRF I guess PPI channels is a more relevant use-case. But then you could very quickly find that you need a more dynamic system that can only detect errors at run-time anyway.
static SENSOR: Mutex<
RefCell<
Option<
Mlx9061x<
I2c<
I2C1,
(
PB6<Alternate<AF4, Output<OpenDrain>>>,
PB7<Alternate<AF4, Output<OpenDrain>>>,
),
>,
ic::Mlx90614,
>,
>,
>,
> = Mutex::new(RefCell::new(None));
If you only use a peripheral struct (eg i2c here) in the main function, the type state system makes sense. If you pass it across function boundaries, or use statically like, this, it may not be. The rust HALs and tutorials that use this pattern tend to leave function boundaries etc (where you need to explicitly declare types) out of examples.
https://github.com/ziglang/zig/commit/0808d98e10c5fea27cebf9...
That's a generic container class (similar to vector in C++ or List in C#). But! With a twist!
It stores structs in "column major" order in memory (e.g., if a struct had two fields A and B, then in-memory layout would be A...AB...B), and you can idiomatically and efficiently get a a slice of the values of each column.
I.e., it's a datastructure that automatically applies the struct-of-arrays optimization:
https://en.m.wikipedia.org/wiki/AoS_and_SoA#Structure_of_Arr...
And the code to do it is straightforward, normal Zig.
Pretty awesome stuff!
This is one way,
http://www.lispworks.com/documentation/lw70/DV/html/delivery...
D doesn't need a successor, rather a better marketing and fixing all lose ends.
Don't know if that actually applies to Zig/D, but that's how I read it.
I use Lisps all the time and love them. But I recognize that most people don't like them. At some point we have to meet people where they are, if we want to have broad impact. Ideas on their own aren't good enough, they need to be packaged up in the right way, approachable to the right people, marketed appropriately, etc. The tech itself is just a small part of what it takes for an idea to create impact.
What zig offers is completely different: it allows you freely manipulate types as data, while disallowing the ability to manipulate syntax.
I like how Julia implemented this (but they use Fortran-like defaults, with clear inspiration from matlab, numpy, etc), with a relatively compact set of functions to do any sort of "index ordering": https://julialang.org/blog/2016/02/iteration/
Rust is very much a child of Ocaml (with a lot of idioms form haskell) with much more control of memory than pretty much any other language (which probably makes it better for implementing complex or safety critical things like optimizing compilers or an operating system). Actually, learning any ML language is probably easier than Rust, but you'll get more fluent at a ML-like (expression based) language like Rust. For me, I felt like I understood Rust way more after looking at how rustc works and started unlearned everything I knew about C/C++.
I am on a countdown to a Rust OS ...
(An OS implemented in Rust).-Ed: I really don't mean this to be snarky, even though looking back it kind of sounds like it. Sometimes you just have to stop fretting about phrasing, slap a smiley on and ship the thing...
Appreciate the comment, on the contrary ...
Redox FTW! (Besides, microkernels, which are nice ...)
(I take it someone from the Redox team saw this and took to the downvotes tough :)
How do you mean? As I understand it, the only difference in these languages in this regard is in 2D arrays, where in C, the current row is in the current cache line and in FORTRAN the current column is in the current cache line. It seems to me you are if anything slightly more likely to want to do several operations on the current row, so the C way is better. What am I missing?
Yes, it makes the language more approachable, yes it makes it easier for people who aren't familiar with the language to understand what's going on. That's nice, but that's not critically important IMO. It's nice if you want to impress people on HN, but if you use the language day-to-day you'll get over that stuff pretty quickly.
On the other hand this type of extreme customization means that even for somebody very familiar with the language you still have to be on your toes because innocuous looking code could behave surprisingly due to comptime shenanigans. On the other hand languages with a more rigid structure may end up being more verbose but that leads to code that a proficient coder can unambiguously understand without having to mentally expand comptime blocks or macros.
This is effectively the metaprogramming equivalent of the statically vs dynamically typed debate. Yes, dynamic code is easier to write but it can be harder to maintain and leads to worse compiler diagnostics and generally requires more unit tests to validate that it's doing the right thing. I think macros/comptime behave similarly when compared to stricter, more limited metaprograming like Rust's generics system.
Rust has macros too of course, but they're a pain to write in my experience, and I'd almost argue that it's a feature. You only use them if you really have to, and after careful consideration, or at least that's how I use them.
Could you give some concrete examples of this?
Unlike C, zig's comptime doesn't have grammar altering abilities
I guess that there is some small truth to your allusion to the static-vs-dynamic debate, but Zig does give you errors at compile-time, and elaborate things it can check at runtime are much easier to express than in Rust. But I would say that Rust is a language in a well-known tradition, and is clearly a "cleaned up C++", while Zig is something that we haven't seen before. It is not dynamic in the same sense as dynamic languages -- you get the checks done at compile-time -- but it is not part of the familiar tradition of typed languages or even any low-level language.
I'm not a devout minimalist, but when it comes to low-level programming in particular, language simplicity is a very important feature, and before Zig it wasn't clear it was achievable at all in low-level languages without significantly compromising on expressivity and safety.
Where they differ: memory safety, sum types (don't tell me std::variant is a valid replacement), move semantics, having pointers, classes, GC vs RAII, statement vs expressions... That's a lot of differences.
What do you mean 'beyond'? It's not like there are many other languages that have compile-time polymorphism. (Java, Go, C, etc., don't.)
> memory safety, sum types (don't tell me std::variant is a valid replacement), move semantics, having pointers, classes, GC vs RAII, statement vs expressions... That's a lot of differences.
ML ignores the real performance and architecture considerations, so yeah, of course it is a simpler and more 'elegant' language. As a teaching aid, yeah, I think all C++ programmers should be forced to program something in an ML-derived language.
But once you start handling the real-world edge cases and requirements you'd end up in a place very similar to C++.
Java does, it's called generics. Also D, rust, Ada, free pascal, nim, and most statically typed languages from the last 3 decades (even Go is finally getting them ). Still can't see why C++ is closer to ML than C, since it's literally an almost compatible superset of C.
No, Java is still pointers to Object and dynamic dispatch under the hood. Generics didn't change this at all.
> Still can't see why C++ is closer to ML than C, since it's literally an almost compatible superset of C.
ATS is also 'literally an almost compatible superset of C'.
I agree that comptime is not the same thing as Rust macros, I mostly mentioned Rust macros because I felt it was a gotcha to my argument since my criticisms of Zig's comptime could be levied at Rust's macros.
To be a little more specific in my criticism, the fact that Zig implements generics with comptime is a bit of a red flag for me. I worry, perhaps unreasonably, that it's going to lead to fragmentation in the way generics are handled in various libraries, leading to headaches and incompatibilities. It is a smart solution, but I wonder if it's a pragmatical solution.
It's definitely an interesting approach at any rate, it's great to see all this creativity in systems language. I don't want to sound too critical of Zig, it's a cool language.
> my criticisms of Zig's comptime could be levied at Rust's macros.
Except that comptime is nothing at all like macros, even though, as it turns out, it can replace enough of their use to make them unnecessary in low-level languages.
> I worry, perhaps unreasonably, that it's going to lead to fragmentation in the way generics are handled in various libraries, leading to headaches and incompatibilities.
But generics in Zig are just functions, and so the problem should be no better but no worse than any API. comptime is drastically less "crazy" or "weird" or hard to make compatible than macros, which Zig doesn't have at all.
I think the idea that zig is a very sharp knife is true, but the overlap of footguns and comptime is not as big as one would think (@fieldParentPtr mistakes comes to mind, but that's sort of about it)
TANSTAAFL. Compile-time evaluation cannot truly "permeate" a language because most practical languages preserve a phase distinction between compile time and runtime. (This phase distinction is somewhat softened, e.g. in interpreted languages as well as in advanced PL's which include such features as dependent types). For a system programming language like Rust which relies on this clear-cut phase separation, macros and proc macros (as well as `const`-marked expressions and functions) are ultimately more elegant.
constexpr is basically making more of C++ available at compile time -- e.g. lots of C++11, 14, 17, and 20 are just allowing more of the language at compile time. Including STL, allocators, etc.
Zig simplifies everything by designing in comptime up front, rather than gradually opening it up with ad hoc rules over 10+ years.
My understanding is that Rust is going down the same path as C++. So you're going to have 2 kinds of macros AND comptime-like/constexpr-like compile time execution.
> Zig simplifies everything by designing in comptime up front, rather than gradually opening it up with ad hoc rules over 10+ years.
I am not 100% sure that I agree with these characterizations, personally, but they're both valid strategies for sure.
> In particular, that much of the complexity I’d unconsciously attributed to the domain — “this is what systems programming is like” — was in fact a consequence of deliberate Rust design decisions.
I also thought that, and, to be fair to Rust, it is following the tradition of C++ and Ada, two low-level languages that would also easily make the top five most complex languages in history alongside Rust. Until Zig showed up, I, and I think many others, didn't believe that an expressive low-level language could be made so simple, certainly not one that values safety.
I like Zig, but can't see anything particularly revolutionary about it's design.
The features mostly an evolution of concepts found in languages like D or Nim.
Between the two, Rust is much more revolutionary. Although Rust, of course, was also heavily inspired by research languages that came before.
There's very little happening in CS that hasn't already been conceptualized in the 50-70ies.
[1]: I.e. if x and y have the same meaning ("reference"), in Lisp -- and in any other language with macros -- you could write a parameterised expression e, such that e(x) does not have the same meaning as e(y); you can't do that in Zig.
Other languages also have more-or-less general partial evaluation constructs, but they're not revolutionary because they didn't realise they can express traditional constructs in terms of partial evaluation. Zig is revolutionary in that its simple partial evaluation construct replaces generics/templates, typeclasses/concepts/traits, macros and conditional compilation. The result is something that is consistent, extremely powerful, and yet exceptionally simple.
It forces you to implement a lot of logic in "userspace" that other languages do for you automatically.
Complexity for certain abstractions moves from the language to user code, at the expense of consistency, cohesion, totality and (auto-generated) documentation.
It will be interesting to see how things play out for Zig once the ecosystem grows a little and libraries appear, but there are very significant downsides to this approach.
[1] http://www.winestockwebdesign.com/Essays/Lisp_Curse.html
So I think it's a fair statement. Rust is safety obsessed. Zig is doing its best while paying top tribute to other gods.
Assumption 2: the more we do with a language, the bigger the language has to be.
I am sceptical about (1), and the only way (2) can possibly be true is if the standard library is part of the language (which it really is not: it's user space stuff, curated approved by whoever's in charge). Don't be excessively pessimistic. It's just as irrational as misguided optimism.
I think the person you were replying to implies that after a certain point (1.0 release?), Zig's userbase will increase to such levels that it can be considered "used" by (many) people
We'll see how it goes. I won't bet my hat on it, but Zig does seem to be on a good path to stay simple even as it matures.
This is what C++98 did, except they called their one true comptime evaluation construct "templates", and they did it by accident. There's a reason why Rust introduced generics and typeclasses separately: C++98 templates as bespoke comptime evaluation was a disaster, and this was clear already in the C++ community.
Right, except not at all, because templates' syntactic elements are distinct from the "object" part of the language, so it is not a partial evaluation construct for C++, but rather a separate (and rather complex) meta-language for C++. In Zig there is just Zig (with its superb error reporting mechanism), and comptime partially evaluates it. Zig distances itself from C++'s problematic design much more than Rust, which, when all is said and done, is pretty darn similar to C++.
But that's the problem with revolutionary design. Your ability to compare it to what came before it is limited because it isn't really similar to anything. Luckily, Zig can be fully learned in a day or two, so there's no need to rely on comparisons for long. You can quickly learn it and decide if it's your cup of tea or not; even if it isn't, you'd have learned something quite refreshing and inspirational, and without spending too much time.
I do agree that there is something more mysterious about Zig. Nobody knows how "good" Rust is yet, either, but it's probably no worse than C++ when we factor all elements that matter to C++/Rust developers, and we're willing to accept that it's also probably not drastically better, except maybe when it comes to undefined behaviour. Zig is more of an unknown because it is so different. It has the potential to be worse than C++, but it can also be much better. At the very least, it is very interesting in that it offers a completely new vision for how low-level programming could be done.
For me, memory and data-race safety and absence of undefined behavior are critical features, but it would be misleading if I were to go around saying "C, C++ and Zig are all pretty darn similar".
The similarity is that they both espouse the very same design philosophy for low-level programming. It's a pretty big similarity.
> For me, memory and data-race safety and absence of undefined behavior are critical features, but it would be misleading if I were to go around saying "C, C++ and Zig are all pretty darn similar".
It would be misleading, because memory safety and undefined behaviour in Zig is much closer to Rust than to C++. Even where it's not the same as Rust, it's still very different from C/C++. Safety and correctness are as emphasised in Zig as in Rust; they just go about achieving them differently. It is not clear at all which of them achieves correctness better.
This is so far from true I cannot take you seriously.
Zig doesn't have any kind of lifetime analysis, so it's as vulnerable to use-after-free/dangling pointers as C and C++ are. That alone rules out Zig from ever being considered "memory safe" in any meaningful sense.
[Yes, I'm aware of GeneralPurposeAllocator, but that is not something you want to ship in production. "Never reuse any virtual address space" is a disaster for the OS (VMA fragmentation, TLB shootdown IPIs) and the hardware (TLBs, caches). That's why no-one ships such a thing in production for C/C++. GeneralPurposeAllocator will no doubt be useful for debugging (though less effective than ASAN or Valgrind) but safety in production is the game here; ASAN doesn't make C/C++ "memory safe".]
Zig also allows data races so Zig programs can have undefined behaviour via data races on non-atomic values. Again, this cannot be fixed.
Even smart pointers (e.g. reference counting) are nasty in Zig. Zig doesn't have destructors so cleaning up an owning pointer or a refcounting pointer requires developers to write manual "defer" statements. Worse, these only work at function scope so you also have to write manual cleanup code for every data structure containing a smart pointer. Without idiomatic smart pointers Zig will likely be more prone to UAF bugs (and leaks) than C++.
Actually you do, because memory safety bugs are more likely to be exploitable than some arbitrary correctness bug, because they can be weaponized to take full control of the program.
The reality is that UAF/dangling pointers are a major source of CVEs in mature software. Rust prevents those in practice, Zig doesn't. You think Zig is going to be much better than Rust at preventing other kinds of bugs. I see zero evidence of that.
What people miss is that a simple a language allows you to apply your smarts to solving the actual problems in front of you instead of puzzling over language features. You can only handle so much cognitive load at once, and ideally the vast majority of that should be devoted to whatever problem you’re solving, not to the language itself.
Ironically, this fact is the same fact that makes complex languages more fun for hobbyists. There’s simply more to explore, and to try, and to solve, when your object isn’t building a product but instead playing with a language. It’s a different purpose, but people very rarely acknowledge this fact, likely because they’d rather pretend their purposes are clearly mechanical and business oriented. It’s ok to just want to have fun sometimes.
It definitely has merit. I've run into this exact same thing in type-system heavy languages like Haskell/Scala/Rust, where I spend more time juggling abstractions than implementing features.
But there is an additional dimension: abstractions often make the first implementation much more cumbersome. But most code is maintained and read much more than it is written, and abstractions can make extending and maintaining a code base much easier.
It's also good to remember that the existence of certain language abstractions doesn't mean you have to use them.
You have to find the right tradeoff.
Personally, I find the "bring your own allocator" philosophy to be pretty radical. Yeah, other systems programming languages can facilitate additional allocators beyond "the" allocator for the language's runtime, but Zig seems to optimize for that case, which makes it a lot more intuitive from a learning perspective (no more guessing about where the memory lives). Even Rust (last I checked) defaults to assuming some global one-size-fits-all allocator.
There's also Zig's flavor of compile-time code / metaprogramming. It's probably less powerful than Rust's macros, but I feel like it's a lot cleaner and intuitive, and I'd argue that being able to run ordinary Zig code at compile time is powerful enough of a feature for Zig's use cases. Ultimately, it's a nice happy medium between full-blown metaprogramming (like in Lisp and - from what I understand - Rust) v. preprocessing (like in C/C++).
And yeah, I'm sure there's plenty of prior art for everything that Zig does, but I don't know of any other languages that combine these things in such a simple and intuitive and principle-of-least-astonishment-friendly way Zig does.
You can substitute whatever allocator you want: https://doc.rust-lang.org/std/alloc/trait.GlobalAlloc.html
Unless you're talking about some other restriction I'm not aware of.
This is vastly more powerful, I have used this to tailor an allocator to specific data structures for better performance.
And really, what you're talking about here is "the standard library data structures," which aren't super likely to be used in firmware anyway. It's a lot easier for ecosystem data structures to add support, after all, they already would choose to call the global allocator, so now they can do either. And it is much easier for them to cut backwards-incompatible changes, if they have to.
BufWrite would do it the same as any data structure would, an additional parameter, all the same.
Why not? Zig's std library is specifically designed to be usable for freestanding/baremetal targets (e.g. firmware), and the compiler is smart enough to only include the parts of a library (including std) that are actually used. If you do need to reimplement a part of std, you can just... reimplement that part, and import your own implementation instead of the one from std.
Unless you're talking purely about Rust?
There is of course nothing stopping a function from ignoring this and using the C global allocator if need be (or, as I've done in some experiments, using a C library's custom allocator - in my case that of SQLite).
(EDIT: from what I understand, there's technically nothing stopping C from using this sort of strategy, either; a struct of function pointers ain't exactly exotic. It's just a matter of libraries being written with that convention in mind, which doesn't seem to be very common.)
Edit: I guess the difference is that Zig doesn't have years of not supporting it, making the ecosystem more prone to support it.
[0] https://twitter.com/SebAaltonen/status/1080235671883841541
- As you mentioned, if I'm editing a document, it's useful to have an allocator on a chunk of memory dedicated to that document. When I close the document, I can then simply free that chunk of memory - and everything allocated against it.
- If I'm implementing an operating system, I'm probably going to want to give each application, driver, etc. a limited amount of memory to use and an allocator against that memory, both so that I can free a whole process at once when it terminates and so that a single process can't gobble up memory unless my operating system specifically grants it more to use (i.e. by itself allocating more memory and making the process' allocator aware of it).
You probably don't want to do this directly. Instead you want to walk the object graph and run cleanup code for everything in that graph, because in general there will be resources that aren't just memory that need to be released, and for consistency with normal operation, it should deallocate memory as it goes.
You probably don't want to allocate an actual "chunk of memory" either. That just creates unnecessary fragmentation. All you really need is accounting and the ability to report when you're consuming too much memory.
Your driver example is not an example where you would allocate memory per software component. You would actually want to allocate per device, not per driver module; it's just confusing because in many cases there is only one device. But if you can plug in many devices that use the same driver, you'd want independent allocation accounting per device.
Zig already handles this with its "defer" feature; as a resource goes out of scope, it can be released automatically. In the document example, that document's existence would likely be a running function, and as that function terminates, it would likely have "defer" statements kick in that free the document's chunk of memory and release any file descriptors and such.
> You probably don't want to allocate an actual "chunk of memory" either. That just creates unnecessary fragmentation.
If anything that should help reduce fragmentation, or at least help reduce its impacts, since you have better control over whether that allocation exists as a contiguous block.
> All you really need is accounting and the ability to report when you're consuming too much memory.
Which is trivial to do when you know for sure that a given component can only work with a given chunk of memory.
But yeah, nothing stopping anyone from implementing an allocator that cares nothing about where its bytes actually live, and just keeping a running tab of how much memory it's used. That is: using custom allocators is an elegant and simple way to implement that accounting, since that's basically what an allocator already is.
> But if you can plug in many devices that use the same driver, you'd want independent allocation accounting per device.
We're probably talking about the same thing here, then, but with slightly different terminology (and perhaps different structure); I'd be pushing for each device to be controlled by an instance of a driver (much like how an ordinary process is an instance of a program), and it would be those per-device instances that would each have their own allocator. Those instances are what I'm calling "drivers" in this context; they might share the same code, but they run independently (or at least they should run independently; a single malfunctioning disk shouldn't bring down all the other disks).
No, that would mean an application managing multiple documents would need one thread per document, which is not normal practice for GUIs. In fact it would then need one event loop per document thread which is not even possible on many platforms.
"defer" simply doesn't serve as a wholesale replacement for destructors, but that's a tangent to this discussion.
> If anything that should help reduce fragmentation
No, there would be fragmentation at document granularity. For example, if you create a document, add a lot of content to it, then delete some of that content, then do that again for several documents, the memory used would be the sum of the maximum sizes of the documents.
I agree with the rest of your comment.
Unless those functions are async, which Zig also supports (even on freestanding targets!). Single OS thread, single event loop, many concurrent cooperatively-scheduled functions. Or you can get fancy and implement a VM that in turn runs preemptively-scheduled userspace processes, in essence basically reinventing Erlang's abstract machine (and this is exactly a pet project I'm working on, on that note).
And even keeping each document in its own (OS) thread ain't really that unprecedented; browsers already do this, last I checked (each open tab being a "document" in this context) - in some cases (like Chrome) even doing one "document" per process.
> For example, if you create a document, add a lot of content to it, then delete some of that content, then do that again for several documents, the memory used would be the sum of the maximum sizes of the documents.
Would that not also be the case if all those documents used a single shared block of memory? Again, splitting things up helps avoid fragmentation here, especially if you know that most documents won't exceed a certain size (in which case fragmentation is only an issue for data beyond that boundary) - or, better yet, if you ain't storing the whole document in memory, in which case the buffer of actively-in-use data can be fixed. Further, if each allocation is a whole page of memory, then that's about as much control over fragmentation as an application can hope for beyond itself being the OS (and probably won't make much of a difference if those pages are scattered across RAM anyway; swapping would definitely suffer on spinning rust, but that's already bad news performance-wise anyway).
That is not correct. (Source: I am a former Mozilla Distinguished Engineer.)
Chrome (and Firefox, with Fission enabled) do one process "per site", e.g. one process for all documents at google.com. (In some cases they may use finer granularity for various reasons, but that's the default.) In each process, there is one "main thread" that all documents share.
> Would that not also be the case if all those documents used a single shared block of memory?
No. Memory freed when you delete content from one document would be reused when you add content to another document.
- Arena allocators, where your code allocates a chunk of memory and then creates its own allocator just for that chunk; when that chunk gets freed, so does everything in it. Handy for short-lived data.
- Using different allocators for different regions of memory makes it trivial to compartmentalize things; you could use the OS allocator (or a straight pointer if you're implementing your own OS) to preallocate chunks of memory, slap allocators on those chunks, and hand those to different components, preventing any given component from bringing down the whole program due to a memory leak.
- It's possible to use allocators for verifying code correctness (e.g. detecting memory leaks, testing code under memory exhaustion conditions, etc.).
The C++ STL has custom allocators (though the initial design was sort of botched; there is a new polymorphic allocator mechanism that aims to fix it IIRC)
For example, Zig does not appear to have any concept of lifetimes, and does not enforce single mutable ownership. As I understand it, Zig does not have RAII, either, so cleanup (with "defer", etc.) is also left as an exercise for the programmer. Zig has arenas, allowing quick cleanup, but seems pretty bare-bones otherwise.
(I was relieved to see that Zig does not allow unchecked null pointers.)
If I am to write a browser engine, I'd choose rust, to write an embedded OS I'd choose zig.
1. Orthogonality is the property of a language that it constitute only a small number of concepts, but exactly the ones you need (e.g. C or Scheme are orthogonal, C++17 is not).
2. A good user experience (e.g. of a Web GUI, but also of a programming language) minimizes the violation of expectations of the user (Ben Shneiderman). "Discoverability" has also been used to describe this.
I agree with you that it's desirable for a language that you may have an intuition "it should be written as something like this..." and it just words. Thanks for calling out "guessability"!
I cannot find the appeal of the current iteration. It's very counterintuitive, which makes it really unsuitable for mainstream programming IMHO. Sure, you could argue that it's not intended for mainstream programming and that we want people to know exactly what they're doing, but then you're basically making the same argument Torvalds did for C.
And then you kinda have to ask yourself ... why?
I was generally rooting for mainstream usage of Rust, but I don't see it happening with the path it has taken. I also don't really hope it will for the same reasons.
As long as it's a good language, why do we care if it's mainstream or not?
Zig might be in a good position here as it has very nice C interop, which lets you leverage the past 30 years of programming history, but it's still got a ways to go before it will be "ready for primetime" from the look of it.
- There are extremely few jobs that recognise it. I'm attempting to learn C++ because of this.
- Documentation can be lacking as there isn't as much demand for it, or people with time to write it. That said, personal support in small communities can be great.
- Smaller library ecosystem.
- Survival of the language into the future is less certain without the financial support mainstream languages have.
I've used Haxe for years despite these points, it's a great language. A language is more than it's engineering though.
Ironically, neutralising network effects like that is perhaps the best way to make sure Zig becomes mainstream, eventually.
Not to lionize andy or anything, but I'm pretty sure that strategies to neutralize these concerns is a deliberate choice in his stewardship of the language.
One of the cooler aspects of Zig's breaking changes is that its formatting tool can automatically update your code to the new syntax.
Rust has evolved a lot, even since 1.0, but all changes were well designed and for the better, in my view.
Before claiming why its counterintuitive I think you should have provided some example to backup such claim.
It's also why I feel that the comparison with Zig is a bit unfair: Zig is not memory safe. If Rust was willing to compromise with this constraint it would remove could remove some of the intellectual overhead for the developer and result in simpler looking, if unsafe, code.
But then it would also destroy the one killer feature of the language.
The borrow checker is a Big Deal™, but even outside unsafe blocks, Rust did not go all the way to perfect safety. Safety remains a spectrum, not a binary choice. The extreme end of that spectrum isn't Rust. It's using a proof assistant to mechanically check the correctness of your entire program.
Still, I think my point about safety being a cursor instead of a switch remains.
Also, Zig comptime is extremely powerful, which means you can do many many things with them, but it makes it pretty hard to understand what's happening: you always need to wonder “what will this code become when compiled”. A bit like with super-macro-heavy C code, or even lisp (even though comptime don't even work the same way macro do so you also need to wrap your head around it). In the end, IMHO it makes it “really fun to write, and hard to read”. This, combined with the lack of memory safety[2], probably make Zig the perfect hacker/hobbyist language, but not desirable for production (being the perfect mirror of Rust).
[1] even though Zig isn't a particularly good example for this, when reading real-world Zig code, there's `@` and unusual keywords (`align` `inline` `try` `comptime`, etc.), and (kind of) static typing. Of course it has a lighter syntax than Rust, but it's not like a dynamically-typed language either.
[2] yes, I know, there are some plans to have some kind of op-in memory-safety thanks to runtime checks, which is better than C's “everything is UB and sanitizer are an afterthought”, but still far away from the “proven safe” situation you get when using Rust. It's pretty sad that Zig didn't want to build upon the ownership framework developed by Rust.
In C++17, the author's issues with trying to do port and pin with different pin types, has a pretty elegant solution in C++.
Here is a toy solution.
#include <iostream>
#include <tuple>
/*
for (port, pin) in &[(P0, 10), (P1, 7), ...] {
port.pin_cnf[pin].write(|w| {
w.input().disconnect();
w.dir().output();
w
});
}
*/
template <typename PinTuple, typename F>
void for_each_port_and_pin(PinTuple& tuple, F f) {
std::apply(
[&](auto&&... p) {
auto apply_pin = [&](auto& t) { std::apply(f, t); };
(apply_pin(p), ...);
},
tuple);
}
struct P0 {
void write(int pin) {
std::cout << "Writing on Port P0, pin " << pin << "\n";
}
};
struct P1 {
void write(int pin) {
std::cout << "Writing on Port P1, pin " << pin << "\n";
}
};
int main() {
auto ports_and_pins =
std::tuple{std::tuple{P0{}, 10}, std::tuple{P1{}, 7}};
for_each_port_and_pin(ports_and_pins,
[](auto& port, int pin) { port.write(pin); });
}
Another approach for that sort of genericity would be a trait that is implemented for each target. This ends up being a more formal/structured version of the above, since it defines the interface explicitly, but is potentially over engineering.
On my phone something like:
enum Pin {
Pin0(P0, usize)
Pin1(P1, usize)
}
I know the above doesn't affect the general point, but I have found lots of people struggling with Rust haven't yet discovered that you can easily combine types in Enums.
+ helps better model the domain: P0 and P1 are already device ports, wrapping doesn't clarify anything,
+ nor does it buy you safety; arguably I'd say it makes it less safe, since the real risk with this sort of code is that you fat finger when copy/pasting between the electrical schematic and the firmware, so by adding extra wrapping you further obscure the pin assignments.
_ => {}There are many things about this project that a compiler can't help me with. I had to read about all of the pinouts from a PDF, draw them on a circuit board, and then map those pins in the firmware.
The code only deals with that last part, and in this particular example I decided that it was safer on the whole for the code to be obvious (easy to read + compare with hardware schematic) than to go through contortions with types to make some things more checkable by computer but less-checkable by human inspection.
The main risk here is not passing the wrong value to this match, it's fat fingering the transcription from the schematic.
The better & harder approach matches a language’s virtues to the problem at hand. Strict correctness doesn’t matter much for a hobby program like this - moving fast and having fun are probably more important here. Expressing that with your tools might mean using Zig, or using rust but being sloppy with allocations and .clone() because it’s fine. Or treating rust like C and using unsafe everywhere. “Making invalid states inexpressible” isn’t that important in a fun side project - unless maybe that’s fun for you!
The right question isn’t “What is my favourite tool?”. It’s “If this project had a soul, how would it want to express itself through code?”
I somewhat disagree. What you're saying is basically "pick the right tool for the job", but more poetically (not a criticism, BTW, I like your phrasing).
But what I think this is missing is that the "right" tool is at least in part based on your favorite tools are.
Or to put it differently, using a tool that may be suboptimal for the job, but which you know extremely well, may be better than using a tool you don't know which is optimal for the job.
Of course, this is much less of an issue for hobby projects, and if one of your goals for a project is "learn new stuff" (a goal I often have for my own hobby projects), then picking the optimal language may be exactly the right choice. You get to learn a new thing while not fighting with the language.
For me, when I want to make a quick UI prototype I reach for JS and Svelte. Because I’m comfortable there, its not worth it to also be an expert at rails, and Php and SwiftUI and C#/WPF. But if the only tool in my toolbox was Rust, or C, or Unity or something, I’d be much worse at prototyping user interfaces. The inverse is also true - if I wanted to write a database but only knew JS, I’d be in for a rough time. You want a home base in each domain.
Then it would be (enum type, usize)
Then also at times where all Pins need handling, the.exhaustive matching can reduce risks of not doing so?
_ => {}They just decided to reimplement from scratch their own different solution. That’s completely fair, but doesn’t allow you to make the complexity argument.
It would be like deciding to start a C++ project without using the C++ std library and arguing that C++ is hard because you had to reimplement std::Tuple...
What are the "nrfXXXXX-hal" crates they use that provide the Port/Pin types then? What do the HALs provide in addition that would have helped?
Here the author states that the hard (error prone) part is not the coding, but the transliteration from the manufacturer's data sheets. So Rust seems to be adding complexity for no gain at all. (Which is completely fair for a hobbyist project for a keyboard firmware.)
Does this mean Rust should only be used for big systems where that mandatory explicitness about complexity pays off? Does this mean Rust perhaps would benefit from a mode where certain modules/functions are type checked in a different way? (Or that would just make the language even more complex for no significant gain during programming?)
I recommend watching CppCon and being amazed at how much work is going into the C++ ecosystem right now. Rust is popular in some circles but the programming world is extremely big.
And HN only represents a tiny viewpoint of the programming world.
Say I need to initialize all the columns as output pins.
But my column pins are spread across two ports
That would be one, or two, instructions to set the appropriate port direction registers. Never a loop. Perhaps the author should give Asm a try if he thinks the language is getting in the way.
You may find this interesting: http://halicery.com/Hardware/Intel%208042%20and%208048/8042%...
You can say that 8-bit ones are always going to be cheaper (you can get some for under $0.03 now) but at some point packaging not silicon costs will dominate. Also programmer productivity is very often worth more than a few cents of BOM cost, which is why we are seeing HLL languages used more often in embedded space even in production settings.
As a side note, the NRF52 listed in article have a 32 bit config register per pin (granted the pin direction is separately mapped as 32 bit word per port).
The same thing was said when C# came out, 20 years ago. "Now hardware is cheaper and software costlier"... "C# is the language of the future and will replace X language (and X++ too)"... "Just add some GB of RAM to the server and that's it, since it costs nothing".
Still, we see everyday new articles on HN about new "low-level" or "system" languages, with focus on performance, speed, safety and whatever, on today's hardware. Even today, there is still a need to write low-level, fast software, trying to squeeze every cycle and bit of RAM. And C and C++ are still alive.
> Also programmer productivity is very often worth more than a few cents of BOM cost.
Guess what would happen if I say my boss that a $0.03 MCU can do the same thing as a $0.30 MCU? What would my boss choose? My mental health, or his pocket?
$0.27 x 1000000 = $270000. That's another Ferrari.
"...every time I use Rust on a new project, I run into some issue that forces me to confront a new corner of the language/ecosystem."
This was the tour de force comment for me. This is how I feel about web programming, Swift, growingly so about Kotlin. I figure you can use Rust as a mutable variable for many of the "new languages" for developers to use.
We didn't have time to write you a short language, so we wrote you a complex one instead. Reminds of laws in Congress these days.
"Wirth’s philosophy of programming languages is that a complex language is not required to solve a complex problem. On the contrary, he believes that languages containing complex features whose purposes are to solve complex problems actually hinder the problem solving effort. The idea is that it is difficult enough for a programmer to solve a complex problem without having to also cope with the complexity of the language. Having a simple programming language as a tool reduces the total complexity of the problem to be solved."
There is really no reason ever to even consider restricting yourself to C when coding for a microcontroller. The ability to do computation at compile time is essential in this area, and C is just fundamentally lacking. While in principle you could do your compile-time work in the makefile, instead, complicating your build process rarely turns out well.
Some people insist C++ is about "O-O" notions of inheritance, virtual functions, and heap allocation; others, that it is about STL and std::vector. All such people are dead, dead wrong. You don't (generally!) use any of that in microcontroller coding, yet you routinely draw upon all the most powerful features of the language and library to make code that is maximally short, fast, small, and correct. These features put the type system to work for you, not just checking but actively making code correct.
Debugging on microcontrollers is hard enough with a language that makes bad code easier to write than good code. In modern C++ (as now also in Rust), when the program builds and links, it is more often than not right. The more you help yourself to correctness with powerful language features, the more often this happens.
While Zig has features to do computation at compile time, it lacks features to help make that computation correct. Such features would depend on a more powerful type system than Zig provides. Debugging bad compile-time computation at runtime, in a microcontroller, is no fun.
Huh? Zig will reject comptime overflows/underflows, and you can write suites of inline tests too if you'd like, so whatever you miss at comptime you can constrain using runtime CI-validation at the location of interest.
Zig does look interesting. I'm still wrapping my head around the consequences of comptime but fast compile times and easy interoperability with C (C to Zig and Zig to C) are all positives. And yes I know Rust can compile to a C ABI.
It's also interesting to me that the Zig compiler only verifies if a particular instantiation of a "generic" is correct, not all versions. This makes it closer in some ways to C++'s templates (which, at the end of the day, are little more than string substitution). Like the functor pattern arose in C++ and was made possible by essentially just stringing symbols together and the compiler telling you if you could dereference a type (if you could ever figure that out from 8 pages of template errors in gcc; clang was better).
Rust it seems also generates a ton of IR and leaves the LLVM backend to optimize it away. I found an article talking about the history of how this evolved from the early days of Servo and Rust.
I know there's an ongoing effort to, say, add constexpr like behaviour to Rust. I worry if we'll get to a point where we are with C++, where we constantly bolt things onto it to the point that the language is incredibly complex, forcing people to use a sand and minimal subset.
But it does seem that the complaining about Rust being a difficult language is happening fairly early in its adoption.
Rust is also in the unique position, being low level as well as functional, which means it will always be alien no matter what programming background you have.
In order to guarantee memory and thread safety, it also adds a few restrictions. Some are fundamental (mutable XOR shared, move semantic) others are/were temporary implementation limitations (like lexical lifetimes, or arrays being second class citizens, or the `impl` keyword being usable only in a few cases). In that regard, Rust has become much simpler since it was released 6 years ago, but it still has some margin for progress.
> Rust is also in the unique position, being low level as well as functional, which means it will always be alien no matter what programming background you have.
Having worked with Haskell and generally being very comfortable with functional programming (whether or not I'm happy about the astronomical complexity of Haskell as a language with extensions is another story) and also having a lower-level background I can tell you that the offerings of Rust as a "functional" language (this is generally so ill-defined that you can call basically anything functional) are extremely slim and that I'd never choose it for any of that.
Tagged unions and `match` expressions might make for a functional language to a lot of people but these things are orthogonal to functional programming. They're nice to have, but to harken back to the original topic; Zig has tagged unions and can `switch` on them to unpack payloads just fine.
Like I said, I think FP is ill-defined but even with that in mind it's a stretch to say that Rust is functional.
Most people, I would hope, would agree that function composition is more an inherent feature of FP and honestly I've never seen a particularly compelling argument for Rust having a reasonably useful solution to that. Stitching together methods is one step, but it only gets you so far and isn't as generally useful as what most ML-descendants would give you.
Me too, but my personal experience when working with other people is that we belong to the minority here.
You cannot be “cool” and “mainstream” at the same time.
Sure, adding features is generally a sign that you're trying to capture/retain users (I recently learned that weak equality operators were added to JavaScript for this exact reason and Brendan Eich regrets it *a lot*), but at the same time it feels like things could be more cohesive than they are.
But yeah, I agree that hype trains lose momentum when things become more common. With that said, I don't really feel like Rust is popular/common enough to lose hype. Basically no one uses it in production in comparison to the obvious alternatives and it'd be a bit odd for it to lose hype so soon.
These complaints made the top of HN today, not 5 years ago (when Rust for embedded was way less mature and convenient), for a good reason. And notice that this person isn't advocating that Rust is too complex and you should keep using proven tech like C or Python for doing real stuff, they are advocating using a brand new experimental language, with a radical new design, whose compiler keeps crashing and which still makes breaking changes every once in a while[1]. This specific article is a really good illustration of the hype train moving on.
> but at the same time it feels like things could be more cohesive than they are.
The Rust team attempt to make the feature as cohesive as possible (and compared to a language like C++, they are doing a pretty good job at it) but nothing is perfect. I'm curious if you have specific examples of features that you don't think are cohesive though. (One example I can think of is the old `macro_rule!` vs procedural macros, but maybe there are others).
> With that said, I don't really feel like Rust is popular/common enough to lose hype. Basically no one uses it in production in comparison to the obvious alternatives and it'd be a bit odd for it to lose hype so soon.
Hype is a multi-level thing: the Rust hype is still growing (it still makes the front page of HN almost every day), but the most avant-garde hackers are moving away from it. That's nothing unexpected.
[1]: I'm not bashing Zig in any way, nobody expect such a recent language to be polished already!
[2]: https://medium.com/code-adventures/farewell-node-js-4ba9e7f3...
I wonder if there's a better way to do this. Perhaps making GPIO peripherals all the same type, with some sort of feature flag for each pin. It would simplify a lot of things.
From the point of view of an embedded developer tired of C/C++, there's a lot of attractive features in Rust for embedded. But I haven't tried to write more than a trivial project in it. I wonder if this is a fatal flaw or just a problem with how the embedded HAL is defined.
Most GPIO defs are just "this is a struct at 0xAABBCCDD"
The tricky part in Rust is how to make things generic across distinct types. Zig's comptime lets you sort of "duck type" it (but with compile-time checking that all of the methods exist, etc.), whereas Rust requires that you explicitly introduce a trait and implement trait methods across the types. The embedded HAL crates do this with extensive use of macros, for example: https://github.com/nrf-rs/nrf-hal/blob/aae17943efc24baffe30b...
This solution makes sense given the constraints of Rust, but there's quite a cost in terms of compiler time and cognitive overhead to understand what is going on.
(Aside: I didn't use the HAL in my Rust firmware, that's a higher layer of abstraction; I only used the PAC crates.)
I think it's a hard problem because every microcontroller is different. A pretty common pattern in C land is to define a peripheral struct (eg I2C, GPIO) which has a 32 bit uint for each register in the peripheral, and then create a pointer to the physical memory address for each peripheral. That means you can write functions which take an I2C peripheral without knowing which one - and so if you decide later to move over to I2C2 it's just a case of changing one variable.
That works because broadly there's very little difference between I2C1 and I2C2, or GPIO0 and GPIO1, in the microcontroller. If they start being very different then you'd have problems with that approach.
As a mere apprentice of 'low-level' programming, I have no intention of learning Rust. It's certainly brilliant though and understanding its underlying concepts is something I aspire to.
I think maybe a good deal of people at the same stage I am will also leave Rust aside as a specialized hard-to-use tool.
But not specialized, even less than C since it contains C.
https://www.oilshell.org/blog/2016/12/05.html
Zig is way better at metaprogramming, but doesn't give you great safety guarantees. Rust is better at type checking, but has at least 3 different kinds of metaprogramming to patch over usability/composability holes created by that rigidity (2 kinds of macros and const contexts)
> OCaml has had more than one macro system, and it appears that they are not done evolving in incompatible ways.
Looks like Rust is following the path of its ancestor here, unfortunately.
Even though I’m only a dozen hours in, I feel like I can already be productive with Zig without an Internet connection.
I loved Nim and considered it to be my second choice. But something pulled me off. Maybe the GC. I still don't understand though why Nim is not as popular as Go. I mean it has good compile times, good performance and the best part, it compiles to C.
Why these languages and not Go or <insert language here>? Simple. Rust, Nim and Zig have `extern` with a C ABI. People don't realize how big this feature is. Two way communication with C.
I was under the impression that the Nim GC was basically entirely optional. Is that not the case? They also added a bunch of other ways to get basically the same thing, did they not? I remember atomic reference counting and so on being presented as basically a drop-in way, but maybe I misunderstood.
Because some people do not consider any "reference counting" to be "garbage collection, I think "automatic memory management" a more clear term.
Regardless, you can turn off all automatic memory mgmt with --gc:none, though the stdlib won't cooperate much. --gc:arc/orc is fairly practical, though still not quite as bug free as, say, --gc:markAndSweep. Nim gives you many options for many things.
Ref counts have a reputation for being slower than mark/sweep/generational/etc AMMs, but (with a lot of compiler assistance/optimizations) they seem at least as fast (and maybe faster) in practice, at least with Nim which usually performs like C/Rust/etc.
[1] https://researcher.watson.ibm.com/researcher/files/us-bacon/...
For example, a binary tree with <65536 nodes could use 2 byte pointers and in-node space overhead might be just 4B and if you had, say, 4B floats as key-values this might be "only" 100% space overhead with 8B per node total instead of C++ STL-like 8B x 3 (parent pointers) + other extra junk overhead. (IIRC, I measured it once as 80 bytes per node in the default `std::set` impl...).
In Nim, one could probably address this kind of situation with distinct types:
type nodePtr = distinct uint16
Since virtual memory dereference needs no extra "deref context", global variables/closures/local context (like an ever present proc parameter) may be needed to fully convert a narrow pointer to a VM pointer (or to otherwise access data). I think that this is all do-able in Nim, the language, but the stdlib has no direct conventions.
You might be able to keep using the rest of the stdlib by swapping out the impl of newSeq or newString to take a named parameter defaulting to some global arena, but call site-specializable to whatever you want and then replacing the `string` and `seq` types which propagate this not quite hidden optional parameter, getting the nice usability/brevity of a global arena with the nice tunability of specialized allocators. A macro pragma might even make propagating the allocator handle to called procs that also allocate semi-automatic (but someone would need to get PRs in to annotate all needed stdlib procs...). I am unaware of any Nim project which has done this yet. So, there could be compiler/run-time bugs in the way or other blockades. And it may not be possible to have narrow pointer types as with my binary tree example (but this may also not be possible with Zig's conventions).
This is all really just a little color on your probably correct conclusion (and a question about how flexible Zig's pointer types can be in its stdlib convention).
EDIT: Oh, and though that Forum asker was not you, you may get better answers than from just me here by making an account there and asking on the Nim Forum.
Ref-counting cannot cope with cycles in data structures. This can be worked around by contorting your code to work around the issue, but is not what I would call an ideal situation (it creates edge cases that more natural code would be able to cope with), simply to work around limitations in the GC (I will claim it's GC, just not as good as other reachability checkers).
These techniques still seem to be called "adjusted/qualified ref counting" rather than "garbage collection". So, I believe terminological problems have muddied the waters yet again. Or else my own incomplete description did { to the extent that differs at all. :-) }
I was trying to differentiate between "garbage collection" (the memory management family of techniques) and "specific species of algorithms for GC" (where I would class "pure reference counting" as one, and various augmentations of ref-counting as several more, as well as stop-and-copy, o the easier side).
As far as "speed" vs "completeness/correctness" goes, in many (maybe even most) cases, if I have to choose between them, I tend to choose "completeness/correctness".
At work, I changed one of our cluster build pipelines from "fast" to "correct" (making it take about 16x longer to complete). Unfortunately, that was as an action highlighted in a port-mortem as one of the root causes of a cluster completely imploding.
Young ecosystem, language still in flux, doesn't have name of big company.
D compiles (DMD compiler) pretty damn fast too. Which is exactly faster to compile D or Go depends on the nod in a horse race.
Google's marketing budget.
> Despite all this, I still don’t feel comfortable with Rust.
This was pretty much my entire experience with Rust. No matter how much I used it I didn't feel comfortable with it. I tried it out when it was still pretty young and it felt like a child of C++ and ML. It is just as feature rich as C++ and some people obviously like that. But I just couldn't keep up with it.
Those of you not in embedded may not get just how stunning an achievement that is from "C works sometimes and nothing else does ever".
And a lot of that has been the Rust embedded guys banging on everything in the embedded space. Language, IDE, debugging, hardware probes, everything.
Even if you're not in Rust, have a cheer for the Rust embedded guys because they are laying down pathways that folks like this can walk through with other languages like Zig.
fn read_keys(port: #[cfg(feature = "splitapple")]
nrf52840_hal::pac::P1
#[cfg(feature = "keytron")]
nrf52833_hal::pac::P0) -> Packet {}
Nevermind that you could actually do this, although you probably shouldn't
fn read_keys(#[cfg(feature = "splitapple")]
port: nrf52840_hal::pac::P1
#[cfg(feature = "keytron")]
port: nrf52833_hal::pac::P0) -> Packet {}> However, I ended up finding these absences liberating — this is where the “fun” comes in.
This just feels like a weird "it's not a bug, it's a feature!" Stockholm syndrome. "I actually LIKE that it doesn't do what I want it to do." Baffles me.
This sentence from the blog post is key:
"For example, it’s now quite clear to me that Rust is a language which has a dedicated feature for everything."
You always have that nagging doubt in the back of the head that you don't know the language good enough yet. And I never got rid of that feeling in over 20 years of coding C++. Turns out, it's not you, it's the language. Switching to a simpler language like C lets you focus on the problem solving again, not solving language puzzles. Zig is truly a "better C" in that sense, because in a way it is an even simpler language than C (while being much more correct), yet it enables fundamental features that C lacks (like comptime and generics).
But you’re solving a puzzle in either case. Either with a solution you created yourself (much more rewarding!) or a premade one you pick off the shelf (higher chance of first time success, often less time involved)
Everything is a trade off.
Or, read another way, "with simple language X I have to reimplement the things that language Y comes with"
I guess it's all a matter of what level of abstraction one likes to work with. For me, re-implementing a "Set" implementation (eg: go) yet again doesn't count as "productive" just because I'm typing typing typing more more more.
Some people assert that they feel "productive" because of all the typing they need to do to implement it, AGAIN. My stance is that HAVING to do that decreases productivity. I'm mis-quoting, but something Knuth said is to think of it not as "lines written" but "lines spent".
I feel that `go`'s recent acquiescence on templates/generics is a bigger version of this, and related to the Stockholm Syndrome I mentioned earlier; this caused much anguish in some, and to me it felt like because having that feature would case people NOT GETTING to type in their bespoke, artisinal implementations of all that stuff that's been solved a million times over. To me that's just baffling.
Using and learning Zig after about 1.5-2 years is many times more productive because I've already found the edges of the language and I'm able to focus more on my actual task instead of debugging my language knowledge. I say this not only in the context of a work task but for things of every scope.
I personally have never felt like I "knew" Haskell, despite working with it and being able to create solutions in it. I know parts of it and there are big parts of it looming somewhere in the distance. There are language extension names that actually give me anxiety when I see them at the top of source files. I think the same feeling is common in C++ users, though admittedly I don't have my finger on that pulse anymore.
Is it possible to create finished solutions that are shorter and where your solution seems to fit the problem more directly in these languages? Yeah, I guess, but at the same time I'm not sure it's ever finished and the anxiety this kind of thing can cause when using these languages I think is underestimated.
There's a tension that this creates where you have to intentionally limit yourself (and suggest others to limit themselves, see "Simple Haskell") because you feel the weight of all of this complexity on you constantly.
"Maybe if we read this 'Thinking in Types' book we can cut down on the amount of lines in our code base, guys?". Meanwhile people using intentionally small languages are provably super productive and you see a lot of coping mechanisms in the communities with these ever-growing colossi languages.
(With all of this said I'm not entirely sure I'd be sold on Zig if it didn't at least have tagged unions and matching/unpacking of them via `switch`. There are language features that I've grown so accustomed to that I feel like I can barely program without them.)
I, however, still like the borrow checker and the dances with ownership Rust has. It eliminates many kinds of bugs.
Just I wish the language was more easy to learn.
fn read_keys(#[cfg(feature = "splitapple")]
port: nrf52840_hal::pac::P1,
#[cfg(feature = "keytron")]
port: nrf52833_hal::pac::P0) -> Packet {}
#[cfg(feature = "splitapple")]
type Port = nrf52840_hal::pac::P1;
#[cfg(feature = "keytron")]
type Port = nrf52833_hal::pac::P0;
fn read_keys(port: Port) -> Packet {}
A more "rustic" solution would be to use a trait, e.g.
trait ReadKeys {
type Port;
fn read_keys(Self::Port) -> Packet {}
}
impl ReadKeys for Splitapple {…}
impl ReadKeys for Keytron {…}Use C! I'm in the process of doing my first "serious" project in 'straight' C (I'm a C++ guy from long ago) and it's taken me a while to get into it properly, but I'm starting to get its philosophy and it's becoming easy.
But also, in the embedded space, it clearly has all the support you could ever want. I'm also gradually falling for Zephyr (https://www.zephyrproject.org) that has all the support for (eg) callbacks, all sorts of low level stuff.
And one more ... the way you'd solve this in Zephyr is to use the scary simple API (https://docs.zephyrproject.org/latest/reference/peripherals/...).
https://github.com/zephyrproject-rtos/zephyr/blob/master/sam...
Often, I do a small PoC which works but then I put it inside bigger project I'm hitting one wall after another. Sure, simple examples work fine but advanced usage almost always break and I don't have any expectations anymore (I will avoid new features next time).
Some examples:
- it's still not possible to write your own smart pointer
which would work with dyn traits, including coercions
- macros have artificial limits when called from other
macros (which is exactly why you can't do a PoC and put it into bigger project)
- borrow-checker is very dumb and even simple things like
x(&mut self.a, &mut self.b) don't work and require you
to do destructures, inners and other mumbo-jumbos.
- often, solution is to write macro, I feel like I'm
writing macros all the time lately
(and macros are entirely different language BTW)I recently wrote a custom HTTP framework in Rust and had spent 2 weeks to figure out how to store a collection of async closure so I can call any of them later. The idea / goal was very clear, but it was so hard for me to figure out how to implement it in the language.
Maybe I'm not smart enough for Rust, but I've already used Rust for more than 1 year, and still struggled to use it. I don't know if I should switch to a different language, e.g. Golang.
Go is so focused on simplicity that you'll be up and running (and have your server working) in like a day, tops. Then you can ponder the tradeoffs from a place of (some) knowledge not hypotheticals.
That said,
1. Just seems like the same lack-of-feature/"simple" stockholm syndrome from Go. To each their own. 2. It would seem that the author could take the structure of their Zig application and almost directly port it back to Rust (e.g., their conditional compilation problems go away if you do it at a high enough level)
`inline for` is pretty special, though. I bet.... now someone has written a published macro for it.
I don't think I'd say "lol no generics" to Zig… Though I'd have to try to really see whether I'm missing anything (and I think I'd notice pretty quick, with my love for OCaml). There's a chance I'd miss closures.
Yup, and this phenomenon has a popular name called 'Rewrite it in Rust' Not sure though, that it has a positive connotation.
> Just seems like the same lack-of-feature/"simple" stockholm syndrome from Go. To each their own.
Huh, I'd say another Rust fanboy obsessed with feature list than working software. You would call it a fair take, right?
He originally wrote it in Rust and solved some of his problems when he rewrote it in Zig in a different architecture. The same architecture in Rust would have solved the same problems.
And no, I would not call your second paragraph a fair take.
Again: `inline for` has been in D for more than 10 years.
I'm especially impressed with the C interop. You can just import C headers, and Zig will use clang to analyze the header and make all symbols available as regular Zig functions etc. No need for manually writing bindings, which is always an awkward and error-prone chore, or use external tools like bindgen, which still takes quite a bit of effort. Things just work. Zig can also just compile C code.
Rust indeed can feel very heavy, bloated and complicated. The language has a lot of complex features and a steep learning curve.
On the other hand, Rust has an extremely powerful type system that allows building very clean abstractions that enforce correctness. I've never worked with a language that makes it so easy to write correct, maintainable and performant code. With Rust I can jump into almost any code base and contribute, with a high confidence that the compiler will catch most of the obvious issues.
The defining feature of Rust is also the borrow checker and thread safety (Send/Sync), which contribute a lot to the mentioned correctness. Zigs doesn't help you much here. The language is not much of an improvement over C/C++ in this regard. The long-term plan for Zig seems to be static analysis, but if the many attempts for C/C++ in this domain show anything is that this is not possible without severe restrictions and gaps.
Choosing to forego generics and do everything with a comptime abstraction makes Zig a lot easier to understand, compared to Rust generics and traits. The downside is that documentation and predictability suffers. Comptime abstractions can fail to compile with unexpected inputs and require quite a bit of effort. They are also problematic for composability, and require manual documentation, instead of getting nicely autogenerated information about traits and bounds.
Many design decisions in Rust are not inherently tied to the borrow checker. Rust could be a considerably simpler, more concise language. But I also think Rust has gotten many aspects right.
It will be very interesting to see how Zig evolves, but for me, the borrow checker, thread safety and ability to tightly scope `unsafe` would make me chose Rust over Zig for almost all projects.
The complexity of Rust is a pill you have to swallow to get those guarantees, unless you use something like Ada/Spark or verifiable subsets of C - which are both more powerful than Rust in this regard, but also a lot more effort.
Some smaller paper cuts, which are partially just due to the relative youth of Zig:
* no (official) package manager yet, though this is aparently being worked on
* documentation is often incomplete and lacking
* error handling with inferred error sets and `try` is very nice! But for now errors can't hold any data, they are just identifiers, which is often insufficient for good error reporting or handling.
* No closures! (big gotcha)
PS: forgot Objective-C, for coding against macOS APIs.
I do loads of server backend in C++. Never felt that I need anything else for this kind of stuff. Sometimes due to client's insistence I did it in other languages but it was their choice.
C (or even C++) functions can be called directly from D. There is no need for wrapper functions, argument swizzling, and the C functions do not need to be put into a separate DLL.
I think Walter Bright achieved this via implementing a full-blown C++ parser in the dlang compiler. A [God-Tier] achievement.
> * no (official) package manager yet, though this is aparently being worked on
It's the next item on the roadmap as soon as the self-hosted compiler is good enough.
> * documentation is often incomplete and lacking
The language reference is good, but for the stdlib it's best to just read the source code. For other miscellaneous learning materials:
https://github.com/ratfactor/ziglings
https://www.youtube.com/c/ZigSHOWTIME
> * error handling with inferred error sets and `try` is very nice! But for now errors can't hold any data, they are just identifiers, which is often insufficient for good error reporting or handling.
It's still under debate and I'm personally in the camp that errors should not have a payload, so I would avoid assuming that it's definitely the preferable choice. We already have a couple of existing patterns for when diagnostics are needed. That said proposals about adding support for error payloads are still open, so who knows.
https://github.com/ziglang/zig/issues/2647
https://github.com/ziglang/zig/issues/7812
Existing pattern: https://github.com/ziglang/zig/issues/2647#issuecomment-5898...
> * No closures! (big gotcha)
It's possible to create closures already (by declaring a struct type with a method (the closure) and instantiating it immediately after), but it's a clunky solution. We also have an open proposal in this space:
How can you argue that not having access to string index where the JSON was unparseable is better than having access to it? I read through issues/2647, and I read through the "existing pattern", and it just seems obvious to me that containing the error information in the error is better than trying to hack it through side channels.
If A -> B, and B returns an error through a side channel, then A can use it and "what's so bad about that?"
But this doesn't seem to scale very well.
If A is refactored to use an intermediate function X, then it just doesn't work. A -> X -> C would mean that...
X cannot use Zig's normal error handling syntax to automatically propagate this error that A is better equipped to handle, unless we're going to further stipulate that A must pass this Diagnostics struct into X which then must pass it into B.
If we now assume that X calls into two fallible functions, B and C, and each of them provide their own diagnostics structs, then X will have to take in two "out" arguments that provide the diagnostics, and every single caller of X will have to provide those two values.
You see how this goes. It just doesn't seem scalable.
Why not take the current design to its logical conclusion of simply having every function return a Boolean indicating whether it succeeded or not, and then require the caller to look into some "out" argument to determine what the error was? Obviously, that would be extremely annoying.
A tagged error union containing the diagnostic error values is just a minor evolution of the current design that brings huge wins for language ergonomics.
For errors that don't need to carry a value, there is no additional cost: they compile and work exactly as they do today.
For errors that carry a value, the size of the error struct is only the size of the largest error value (plus the tag, of course), not the product of all possible error values, so the global error set will never grow to be enormous unless you have some very weird error type. In which case, you can solve this problem by fixing that error type.
So, I'm not deeply versed in Zig, but I have personally argued in favor of Zig's async design, which seems exceptionally interesting. I had not realized until now how limiting the error system implementation was, but I had superficially appreciated how much less verbose it was than Rust's where you tend to hand write these error enums, or use a bunch of code gen. However, errors benefit tremendously from having the ability to supply payloads.
No one is required to act on the payloads within the errors, but no one is able to if they don't exist.
I think it's not trivial to find a good alternative to status quo.
I believe the solution has been clearly presented to those who are listening, and the downsides are so much smaller than the status quo.
If people on the language team are unwilling to provide developers with the tools to make more robust software because of something like 64 bytes of stack memory... I find that pretty shocking. And yes, I mean that it is extremely difficult to diagnose and fix issues in production software when you only get back a static error value that lacks the dynamic error context that a payload would provide. I’ve been there, done that. Do not want that again.
Such a strong viewpoint (avoiding such error payloads) could at least focus more on the technical aspects of how best to implement the elision of unused error payloads instead of just broadly opposing the concept, since it should be obvious how beneficial those payloads are, and the only question is how to remove them for developers who literally cannot spare dozens of bytes of stack memory. (Which I find hard to believe outside of AVR or PIC microcontrollers, which is probably not the best use case to be optimizing for in a new language.)
Elision is an optimization that can be added later. It’s not obvious that it can’t be done, and there’s no clear reason that it has to be solved first... I just don’t think anyone would actually care enough to implement it once they see how nonexistent the negative impacts are, but it would be a cool optimization.
But, maybe everyone who upvoted the apparently most upvoted GitHub issue on the Zig repo (including myself) is just completely wrong and this obvious solution is actually secretly terrible. It’s entirely possible.
Your comment has not really done anything to help me believe that I’m wrong, though, unfortunately.
But, as I’m basically “no one” in this context, my opinion probably doesn’t really matter.
Note that this isn't like go's "if err = nil" monstrosity, either.
I already addressed how you’re apparently giving up all of the benefits of Zig’s error handling system to do this. That level of convention breaking isn’t a good sign for anything. The current convention really is that bad, from my point of view.
The obvious solution’s extremely tiny overhead could “easily” be avoided in the almost non-existent cases where it matters.
I want robust software by default, not software where I have to use side channel hacks to get the information I need by default... but maybe that’s just me. (And everyone else who upvoted that GitHub issue)
Of course Zig programmers would be unlikely to see the issue here — naturally the people left are the ones who don’t see the problem. People like myself who know from past experience how problematic not having error context is are likely to just avoid Zig until it meets our minimum requirements.
That’s not a problem for existing Zig users — it works for them — but it is annoying to people like me who think Zig otherwise has a number of interesting aspects.
It's possible that erlang developers are merely internalizing sone pain, but it's also a robust system that people have been developing highly reliable and broadly used (e.g. rabbitMQ) systems architectures in, so the choice can't be all that bad.
Side channels are not really a feasible implementation option.
So you have to resort to maintaining the logical information on intermediate levels, returning a result sum type, or stick to good old C paradigms and use a output param. Which negates all value that error sets offer.
I understand that it's not trivial to implement, but for me it's a sort of must to avoid ending up with messy APIs.
From the developer’s point of view, the values would always be there, just unused. The compiler would just make them disappear at compile time if unused.
This quote is from your GP post, and I read this as arguing against having the mechanism for getting this payload in the language (and hence the compiler itself), so my language was intended. Am I misunderstanding?
(This could just be a quirk of my own vocabulary)
You wouldn't have to rewrite the world, but the language is young and maybe more effort could go into making using this pattern more idiomatic and encourage library writers to do it more often.
A package manager is more or less expected by developers now, and I bet you will see a lot more adoption once it's easy to publish and consume libraries with an official packager manager and online repository.
I love Scala because of this as well, and put up with the slow compile times and large runtime needed because I very much value that correctness. I get a lot of the same with Rust (and more, like data races being a compiler error), with the added bonus that so many of its abstractions are zero-cost.
The more I build software, the more I want strong type systems that I can lean on to help ensure correctness. Obviously that won't eliminate all bugs, but building reliable software on a deadline turns out to be really hard, and if a compiler can tell me I'm doing things wrong before it becomes an expensive mistake in production, that's worth the added effort it takes to write in a language that can help me in this way.
It seems like Zig is approaching it from the other side: a "better C". I don't really want a better C; I want a Scala that runs with the CPU and memory footprint of a C program. Rust is probably as close as I'll get to that.
Unfortunately I've found that a lot of developers -- especially senior ones -- don't want to learn anything new, and want to keep churning out the same overengineered, overabstracted, exception-oriented, mutable-spaghetti Java code, year after year. Reminds me of the saying that some senior developers have one year of experience, repeated ten times.
If I understand correctly it also means that Zig will never support inference of generic type parameters. Not a dealbreaker, but unfortunate.
I also wonder how relying on comptime for all these things will affect IDE support. I guess the IDE will need to run these comptime computations a lot.
That said, Zig does look very cool and I think its design choices make a lot of sense. Rewriting from C to Rust is a way bigger step than from C to Zig and you still get a lot of safety improvements (though not quite as many as the borrow checker can provide). It'll be interesting to see how it evolves.
https://clips.twitch.tv/RockyLivelyChimpanzeeDoubleRainbow-P...
Full video available here:
I have an example of that in this blog post: https://kristoff.it/blog/what-is-zig-comptime/
It makes sense to me that `comptime for` means that the `for` is `comptime` but the body of the loop isn't.
The majority of uses of it in unrolling duck-typing cases will most likely be replaced by [1] which allows the same but with `switch` instead. Thus `inline for` will be left for loop unrolling and possibly in use with `inline switch` where it matters.
Maybe more helpfully, Zig is much closer to your "C with better syntax" than Nim. Nim is more like Ada & Lisp had a baby with better/more Python-ish syntax ("better" being subjective, as with most things).
( EDIT: But really all should make their own determinations: https://nim-lang.org )
Absolutely if you're not interested in the safety Rust brings, a lot of the design isn't going to feel worth it. The safety is my favorite part of Rust, though :) Zig is quite explicitly not a safe language.
https://play.rust-lang.org/?version=stable&mode=debug&editio...
There are various conference presentations about it that show it in action.
I think it's simply an outdated oversimplification in 2021 to say that microcontroller projects "normally" don't want heap allocations at all.
TinyGo also makes it relatively easy to avoid heap allocations because you can change a compiler flag to make heap allocations a compiler error^1, if that's required for a particular project.
^1: https://tinygo.org/usage/important-options/ (look at -gc=none)
Another interesting link: https://tinygo.org/compiler-internals/heap-allocation/
For performance-sensitive areas where deterministic latency is critical, a GC simply won’t cut it, even if you can control when to run the GC step.
For lower end microcontrollers, I highly doubt any of those solutions would ever work: there is simply too much “magic” (read: overhead) involved.
This is primarily why C is still king of the microcontroller world.
Go gained some simplicity by not having generics, instead simply special-casing the two most common generic datastructures: lists and maps. But in the end, the desire for generics was too strong, and now they're adding them to the language, and losing that simplicity win.
Zig gained simplicity by not having generics, but instead giving you comp-time evaluation, which can do everything generics can do and more (e.g., replaces need for preprocessor, need for minilanguage for build variants).
I do think both languages have similar philosophies and feel, though Zig is much lower level, and seems to have more of a focus on "get things exactly right" vs. maybe more of a "eh, good enough / hacky" approach from Go.
I wonder what a language that tried to combine this focus on smallness/orthogonality with a borrow checker would look like. (Would it even be possible?)
I realize it doesn't have the same memory safety guarantees as rust, but what I don't quite understand, is what sort of danger that means I'm inviting these days. If my app doesn't deal with anything particular sensitive, do I need to worry about it too much? Am I inviting trouble for my users?
I read Smashing the Stack for Fun and Profit back in the day, and I know that at one point, poorly behaving low level programs could escalate privileges, reach into other programs' memory and so forth.
But is that still relevant these days? I'd think OSes have gotten better about sandboxing programs and such. I know I've seen misbehaving programs segfault before, which I thought was the OS protecting against wayward memory access.
And "memory safety" really doesn't apply if targeting wasm, right? In that case Zig could really shine, since its great for cross compilation and really has memory allocations handled well, and doesn't need a GC, without the "memory safety" downsides.
I guess my question is how much rust's "memory safety" is kind of an XY problem? In what cases is that actually what I should be asking for, compared to the more general question of program correctness? I can see it in rust's original use case of a web browser, since, e.g., tabs shouldn't access each other, but does it apply to all "system" or low level programming use cases?
https://alexgaynor.net/2020/may/27/science-on-memory-unsafet...
You would have to be very aggressive about making the partitioning as fine as necessary, otherwise memory unsafety leads to privilege escalation. For example if memory unsafe code in a single sandbox can download arbitrary PDFs and render them, then a malicious PDF will be able to scan scan local networks and exfiltrate data.
This is something I've thought a lot about as I've started to reach the "hundreds of hours" of working with Rust mark. Besides attributing the complexity of Rust to the domain of systems programming, I think a lot of Rust's complexity often gets attributed as a trade-of you're making for the safety guarantees afforded by the borrow checker. But I think a lot of the complexity in Rust is not related to the borrow checker at all, and is just a result of certain design decisions in the language.
For instance, taking the module system as an example, in general, I can declare a dependency in mu `Cargo.toml` file like this: `"crate_name"`, and then import it into a given source file using the use declaration: `use crate_name`. However there's a special case, where if the crate name uses dashes, like `"crate-name"`, then the compiler will implicitly resolve to that from a use declaration using underscores: `use crate_name`.
Similarly, if wade into an unfamiliar code-base, and I see a use declaration like this: `use path::to::foo", if I want to look for the code for this, it could be in one of three places:
1. The file `src/path/to/foo.rs`
2. The file `src/path/to/foo/mod.rs`
3. Some other file, based on re-export via a `pub use` declaration.
So in order to use Rust effectively, I have to just sort of know about all these implicit behaviors of the compiler, and in my experience it took months to learn enough of these tricks and corners to really just be able to sit down with an idea and start coding in Rust without consulting documentation and examples regularly. And even after that the compiler still surprises me sometimes. To give another example of somewhat vexing implicit behavior which does relate to the ownership system, just today I had a block of code which looked like this:
let x = some_value;
match x { ... }
x = some_other_value;
match x { ... }
let x = some_value;
match x { ... }
// x = some_other_value;
match x { ... } // <-- use after move
I wonder if part of this has to do with the fact that Rust seems to appeal to a certain type of programmer who is attracted to esoteric topics and arcane knowledge, so the fact that Rust is essentially an unlimited well of complexity is more a feature than a bug to them. But I have been thinking a lot about what a programming language would look like which has an ownership system like Rust, algebraic types, and a trait system, but puts a ruthless emphasis on productivity and eschewing complexity.
How do you propose it should work instead? Disallow using match with owned values, so that match never moves? Disallow assigning to a variable where the value has been moved, requiring a new or shadowed variable instead? You could do either of those things, but neither would remove complexity, just move it elsewhere and perhaps cause some new issues. It's easy to complain about complexity, but almost all of it exists for a reason, and hasn't been added just because Rust programmers are "attracted to esoteric topics and arcane knowledge". The implicit behavior you are talking about in this case is the ownership system.
But that's how it already is! The variable x is conceptually a single memory location within the stack frame. The match statement doesn't own the variable x, it owns its previous contents. It has effectively removed whatever was in x and whatever remains in x's memory location is no longer accessible, but the variable is still there.
I don't think this will work well in practice. For example, would you want to declare a new variable every time an array is reallocated at a new memory location?
Reusing variables has both ergonomic (don't have to think about and manage new names) and conceptual (x may represent the same "thing", e.g., a reallocated array) values.
And in this particular case, the compiler will very helpfully tell you what went wrong, so it's not like a programmer will have to hunt down the bug for hours trying to understand what happened. The programmer has to learn this once. Can we then call this a complexity issue then?
> I wonder if part of this has to do with the fact that Rust seems to appeal to a certain type of programmer who is attracted to esoteric topics and arcane knowledge
I find this an unfair take, especially having been a witness to the design process in Rust, where a lot of emphasis and effort is put into coming up with designs that have the right complexity-usefulness balance.
A better way to put it is that Rust matches the values that I care about in a programming language [0][1]. What I love about Rust is that I can rely on it to point out a wider class of mistakes that I'd often make in other languages. Forgetting to deallocate a pointer (C/C++) or using multiple objects when trying to run a synchronized code (Java) often need non-trivial time to debug that ultimately don't teach me anything other than to be more careful. In Rust, I can offload that cognitive load to the compiler. And I find that delightful and satisfying. If it compiled, it is highly likely to be correct, moreso after a refactoring session spanning the entire project. And all this makes the effort in learning whatever complexity Rust has worth it. And I say this with the knowledge that the Rust team is doing their best to address the complexity concerns seriously.
> what a programming language would look like which has an ownership system like Rust, algebraic types, and a trait system, but puts a ruthless emphasis on productivity and eschewing complexity.
Do give this a try. It is likely that you'll have to make different tradeoffs. Or you'll discover novel ideas. Either ways, it'll be a learning experience.
[0] https://www.slideshare.net/bcantrill/platform-values-rust-an... [1] https://www.infoq.com/presentations/rust-tradeoffs/
Unfortunately true in my experience. The two engineers I know have advocated strongly for Rust at my company cared more for the technical elegance of their code than the long-term costs of using said code.
So, I’ve seriously been considering designing a language with the same assurances, but with different mechanics. One that is easier to learn and use due to the consistency of syntax patterns used across the breadth and depth of it.
The creator of Zig has expressed some disregard toward proper security in the current prototypes. The standard library is riddled with DOS vulnerabilities as of right now, and reports for them are closed as "The standard library will be rewritten." or the like.
Take caution.
> microcontrollers have limited program space, roughly 10–100kB in my case
The standard Go toolchain cannot compile for microcontrollers, so the size of binaries that it produces is irrelevant.
Just like there are many compilers for C, there are multiple compilers for Go with different priorities.
Short keywords improve readability in a big way, IMO. There's obviously a balance to strike but I think things like "impl" "const" or "mut" or "ref" are pretty obvious.
> Despite all this, I still don’t feel comfortable with Rust. It feels fractally complex — seemingly every time I use Rust on a new project, I run into some issue that forces me to confront a new corner of the language/ecosystem. Developing my keyboard firmware was no exception: I ran into two problems, and each required learning a completely new language feature.
There must be a balance between features, security & robustness, speed and easy of use. It all lays down to language design, and I guess we need a new language design paradigm, not a programming paradigm, but a new design paradigm on programming languages
Tbh, i think it's a great strategy. As wonderful as it is, zig has enough bugs in it still that you don't want total newbies all in yet, and this acts as a bit of a gatekeeping device (which will be lifted in due time) and also keeps negatively predisposed people "looking for excuses to not use zig" out -- as I'd say works really well in your case!
Take programs in an already popular language and transpile the code to your language.
One of my projects py2many is exactly that. It supports 6 languages now for a small set of features. Would love to review patches if someone submitted a Zig backend.
> Recursion is a fundamental tool in modeling software. However it has an often-overlooked problem: unbounded memory allocation.
That it not necessarily true. It depends on the implementation in the given language. Perhaps in Zig it has that problem, but it is not an inherent property of every recursion one can think of.
If Zig manages to get optimized tail calls like I can enjoy in Scheme, it has a good chance of being the next language I learn. As noted in the next paragraph of the docs, it is still an area of experimentation.
It is necessarily true if you allow non-tail recursive calls.
So it is not necessarily a true statement about recursion. It is only a true statement about non-tail recursive ones. It is an over-generalization. The docs would to well to distinguish those.
But, yes, it's true that you can express some recursion with fixed memory size. And, in fact, the @call[0] built-in has an `always_tail` which asserts that the call should always be generated with tail call recursion optimization, and is an error at compile time, if not.
You might also be interested in following this (open) GitHub issue[1] that explores recursion in more detail.
But, yes, this area of Zig seems to be still a little "experimental", as you say. (But shows great promise, I think!)
[0] https://ziglang.org/documentation/0.7.1/#call [1] https://github.com/ziglang/zig/issues/1006
Edit: Tried with different optimization levels, it seems it might. I wonder if it's an optimization from LLVM. Nevertheless, I don't think that Zig is the kind of language where you want to use recursion as your primary mean of looping; it's not a functional language, it's a procedural language.