Interview with Zig language creator Andrew Kelley [video]
youtube.com
youtube.com
This is a huge advantage over C++ and Rust, because it makes it much harder for e.g. the intern to write code that repeatedly creates a vector or dynamically allocated string in a loop. Or to use something like std::unordered_map or std::deque that allocates wantonly.
True. On the other hand, Zig makes a deliberate decision not to bother itself with memory safety too much, so its a win some, loose some sort of situation.
This is not true. Zig places a strong emphasis on memory safety, it just does so in a way that's very different from either Java's or Rust's. I wrote more about this here: https://news.ycombinator.com/item?id=24293329
This isn't “memory safety”, with this reasoning you could say “C is memory safe if you use ASAN during debug”: it is exactly equivalent except Zig checks are less powerful than the full suite of existing sanitizers for C, but it's enabled by default in debug mode, which is nice.
Which means you cannot guaranty your program doesn't exhibit memory unsafely unless you stumble upon it during testing. Yes there are fewer footguns in Zig[1] than in C (which is the opposite of C++), but dangling pointer dereferences, double free and race conditions will still be lurking in any reasonably-sized codebase. Calling it “memory-safe”, is dishonest. And I actually don't think it serves Zig. It's a clever language with tons of good ideas, no need to oversell it with misleading claims about memory safety.
[1]: but still plenty of them https://ziglang.org/documentation/master/#Undefined-Behavior
Right, if you disable those checks.
> Calling it “memory-safe”, is dishonest.
The language is (or will be) memory safe, at least its safe parts -- C/C++ aren't. True, that safety can be disabled selectively. Yes, there can then be undefined behaviour if you disable that safety, but so can Rust when you disable safety selectively. The design is just different. Just remember that using a language with sound guarantees is no one's goal. The goal is to produce programs without bugs. Sound guarantees in the language is one approach toward that goal; there are others.
Generally, guaranteeing that some set of bugs cannot happen does not necessarily guarantee fewer bugs overall; in fact the opposite might be true. Nobody knows if Zig's strong safety story yields to more correct programs that Rust's, but nobody knows the opposite, either. There are good arguments either way but little data. In any event, Zig is much safer than C, even with ASAN.
We've been going full cicle here, so I'm not interested in spending more time in this conversation.
I'm not sure why we desperately need to classify languages into "memory-safe" or "not memory-safe". The fact is that all languages have various levels memory-safety (sun.misc.Unsafe anyone?) and I do think that Zig deserves recognization for addressing it heads on:
- There's a separate ReleaseSafe optimization level which has all the optimizations, but remains safe. If you care about memory-safety (which most people do!) then this should be your default production build.
- The documentation is very clear about what can cause undefined behavior in ReleaseFast.
- You can override the safety-level (both ways!) for a single scope. If you have something performance critical that can't be expressed in safe Zig you can opt-in to potential undefined behavior. If you have something safety-critical you can opt-in for correctness even though the overall program is compiled with ReleaseFast.
How does those runtime checks kill UAF?
TBD :)
But here's one way that's currently being tried: https://github.com/ziglang/zig/pull/5998
Regarding https://github.com/ziglang/zig/pull/5998 here we're exactly in the realm of C, changing the allocator with a custom one with additional bookkeeping to check for memory management issues. But it tanks performances so you can't generally use it for production (and if you were in a situation were you'd do it anyway, you'd be better off with completely automatic memory management: AKA a GC).
Does it provide safety in the same manner Rust does? Absolutely not. Does it provide less correctness overall? Maybe, and maybe it provides more correctness, and maybe the same. It's hard to say without an empirical study. The problem is that sound guarantees often come at a cost -- for example, to language complexity and compilation speed -- that can have a negative effect on correctness.
Regarding the PR you just sent, I'd like to hear why you think it cannot be applied to C?
> So you make Zig safe, test it, and then remove the guardrails from the performance-critical bits after you're satisfied with their correctness.
This isn't safety… This is “we didn't find any memory issue while fuzzing the software” and you'd get the same guarantee: if your fuzzer didn't cause the memory issue, then it remains in your code in production, waiting to explode one day with some hard to debug Heisenbug that only occurs once in a million…
I didn't mean that a safe allocator cannot be used in C; I meant that C cannot be made memory safe in its entirety as simply as Zig can. Why? Because C has pointer arithmetic while (safe) Zig doesn't, Zig has slices while C doesn't, and C has non-typesafe casts while safe Zig doesn't.
> This isn't safety… This is “we didn't find any memory issue while fuzzing the software” and you'd get the same guarantee:
No, it's not the same guarantee. Fuzzing a C program will not find all the undefined behaviour that fuzzing a Zig program can, for the reasons I mentioned.
It is true that if you use unsafe Zig, i.e. turn off safety for a whole program or some sections of it, you lose the guarantees that safe Zig gives you, and unsafe Zig is indeed not safe (neither is unsafe Rust). But because of the way it's designed, Zig has a way of improving correctness even when safety is removed. This is a tradeoff for sure, but so are sound guarantees, that can have other negative effects on correctness.
Their is no such things as unsafe and safe Zig. All Zig is unsafe, but you can add additional runtime checks (disabled by default in optimized builds) that will slow down your program when used. Using a specific allocator to detect UAF is something you may do in development, but almost surely never in production. And without it your code isn't memory-safe.
> Fuzzing a C program will not find all the undefined behaviour that fuzzing a Zig program can, for the reasons I mentioned.
Zig will have less UB than C, but there will still be lurking UB in your programs no matter how long you test it. Consider the following snippet (on mobile, so this may have stupid syntax errors):
test "this is UB, but the test won't show it" {
const allocator = std.heap.page_allocator;
var buf = try allocator.alloc(u8, 10);
ohNo(allocator, 42, buf);
allocator.free(bar);
}
fn ohNo(allocator: *Allocator, foo: const u8, bar: *u8) void {
if (foo == 1337) {
// double free awaiting to happen in production
allocator.free(bar);
}
}
If you never explicitly test the value “1337” during you debug session, you won't trigger the UB and you won't know it's here, then when you ship your optimized build in production, you'll ship a program with UB in it.Zig is meant to ultimately give you full memory safety, that you can selectively turn off. In addition, there are specific unsafe operations -- clearly marked -- such as casting an integer to a pointer or other non-typesafe casts.
A code unit with safety checks on and without unsafe operations is what I call "safe Zig."
> And without it your code isn't memory-safe.
This is simply not true. Perhaps you mean that you don't have a guarantee that your code is memory-safe, but that's not the same thing.
Our goal is not to write in a language with certain guarantees but to write programs with certain properties, say, without buffer overflows. One way of achieving such a program is to write it in a language that guarantees no such error can happen. Another is to write it in a language that guarantees no such error can happen in development, do some testing, and then remove the guarantees. In the second case it is true that our confidence in the lack of such errors is lower than the first, but in each case it is not 100%, and because the static guarantees are costly, it is possible that the second approach is even more effective at getting to more correct programs overall. They're both common ways for achieving the same goal.
As someone who works with formal methods, we do these tradeoffs in formal verification all the time. It is simple false that sound guarantees are always the best way to correctness -- it would be if they were free, but they're not.
Once you realise that the goal is achieving some desired level of confidence (which is never 100%, as that cannot exist in a physical system anyway) about overall program correctness -- which includes both "safety" and functional properties, each further divided into degrees of severity -- you see that there is no obvious way with the best effectiveness at achieving that goal.
> If you never explicitly test the value “1337” during you debug session, you won't trigger the UB and you won't know it's here
But here, again, you are looking at something in isolation. Because Zig is a simple language, the chances of such paths existing without you noticing are lower; also, because the language is simpler it is easier to write concolic testers that would automatically detect this.
In fact, if such a "rare path" exists in a complex language that causes some functional bug -- ultimately, we don't care what bug breaks our program or leaves it open to security vulnerabilities -- there's a smaller chance that it will be discovered. Which is exactly what I mean by soundness coming at a cost. It guarantees the lack of certain bugs, but because it complicates the language, it can make other bugs more costly to detect.
“But people can write correct C code”. Correct Zig != memory safety. It's the opposite: MEMORY SAFETY IS THE GUARANTEE that your code won't have memory error no matter how broken it is!
> Another is to write it in a language that guarantees no such error can happen in development, do some testing, and then remove the guarantees.
That's what the same kind of design as C is with ASan, TSan, MemSan etc. Yes Zig is less broken than C, leading to fewer sources of memory issues, but for what matters most (Double Free, Use After Free[1], Data Races) Zig and C offers the same level of safety guarantees: none.
> As someone who works with formal methods, we do these tradeoffs in formal verification all the time. It is simple false that sound guarantees are always the best way to correctness -- it would be if they were free, but they're not.
This is a straw man: comparing compile-time enforced ownership (Rust borrowck) to formal method doesn't make any more sense than comparing static typing to formal methods. It adds a lot of learning friction, but that's it. I just grepped my current 90kLoc rust project. You know how many lifetime annotation ('x) there is in it? Fifty-four! Which is one every 1666 lines. Please tell me again how much it cripples productivity and the ability to write correct code!
> Because Zig is a simple language, the chances of such paths existing without you noticing are lower;
If you ever try to use shared-memory parallelism, this kind of bugs will be everywhere! That's simple: every call to allocator.free is a minefield.
> ultimately, we don't care what bug breaks our program or leaves it open to security vulnerabilities
Memory safety issues aren't just security vulnerabilities, more than anything they are horrible bugs to track down, and it costs tons of money.
> Which is exactly what I mean by soundness coming at a cost. It guarantees the lack of certain bugs, but because it complicates the language, it can make other bugs more costly to detect.
This is BS. It's not because a language has few symbols or a simple syntax that it is easier to debug. Otherwise brainfuck would be the ultimate productivity tool. Semantic is what matters, and because it has UBs, Zig semantic is more complex than most languages out there. That's why C is one of the most complex language ever in practice, even if it's really “simple” and easy to “learn”.
Again, don't get me wrong, I have nothing against Zig and I find it refreshing because it has tons of cool ergonomic tricks (and having a built-in sanitizer which “just works” out of the box in debug mode without any other programmer intervention is cool!). It's a nice programming language experiment that will probably inspire a lot of others, and it's probably a really cool language for C programmers who like to manage their memory by themselves and don't want the “nany compiler” Rust has and still have a language with a modern look and feel: that's totally legit.
But memory safe, it isn't.
[1]: which cause more than 30% of Google and Microsoft security issues by itself! (https://www.zdnet.com/article/microsoft-70-percent-of-all-se... https://www.chromium.org/Home/chromium-security/memory-safet...)
I think you have a missing piece of factual information here. Safe Zig (which is not "correct Zig") guarantees (or will guarantee) memory safety everywhere, no matter how broken the code is, as long as you don't use unsafe operations -- just as in Rust. Instead of eliminating some issues at compile time, it does so by panicking at runtime.
> Please tell me again how much it cripples productivity and the ability to write correct code!
If you think that you -- and your 20-person team maintaining a project for 20 years -- can be as productive in Rust as you can in Zig, then Rust is for you. That's not the case for me (maybe it's not a universal thing) and I don't think that would be the case for my team. Personally, I think that universal truths in programming are rare, and I think it is very likely that Rust might be more effective for some and Zig more effective for others, even if you only consider correctness. I'm not trying to convince you that Zig is better than Rust for you; I'm just saying that Zig is better than Rust for me.
> Zig and C offers the same level of safety guarantees: none.
I'm afraid you're simply mistaken, and repeating the same assertion over and over does not make it more correct. Safe Zig will give you the same guarantees as safe Rust except data races.
Once you turn safety off, you don't have a guarantee but you also don't have anywhere near the same level of confidence as you do in the safety of the C program. So the choice is not between 99.999999% confidence of a guarantee and, say, 50%, but there's lots in between, and Zig is in the vicinity of where Rust is -- don't know if better or worse -- but is much better than where C is. Correctness is simply not a binary position.
You accept this position yourself: when you run your Rust program, you also have no guarantees about the overall functional correctness of the program. You still don't think that you're in the same position as everyone else with no such guarantees, right? That's because there are lots of other activities needed to be done to increase confidence in correctness, and so there can be a very, very wide range of correctness within that "no guarantee" which is where we all are in most cases. I think that Zig makes some of those activities easier than C++/Rust, at least for me.
> But memory safe, it isn't.
Except it is, actually (or, rather, will be) because it guarantees no memory safety errors.
Anyway, thank you for your insight. I've been a professional programmer for nearly 25 years, working on large, long-running projects, some of which are very safety critical (many people would die on failure), some employing formal verification, and it is my opinion that Zig's approach to safety is at least as good as Rust's. It's certainly possible that my opinion is shaped by my personal experience. My software would have killed people either due to a buffer overflow or due to an incorrect logic. Sacrificing things that for me would increase the effort to prevent the latter only to increase my confidence that I don't have faults of the former kind from 99.99% to 99.99999999% doesn't seem like a good tradeoff.
While your opininon to the contrary is just as legitimate, barring empirical evidence, you won't be able to convince me. That the most effective approach to increasing correctness is by eliminating an important class of bugs at the significant cost of language complexity and that there is no more effective approach is an interesting hypothesis, but one that is far from being established. I understand why some might believe it to be true, and also why some believe it to be false. Ultimately, safety and correctness are central design goals for both Zig and Rust, but they each make different tradeoffs to achieve what they consider to be a good sweet spots. Not having any definitive evidence over which is "better" in that regard, we must make our languages choices based on other criteria.
- estimate that memory safety isn't paramount (you're not alone in this case, and it's usually the kind of discussion that happens over and over on Rust threads).
- and use the “Zig is memory-safe” as a marketing argument.
“Zig isn't memory safe but we think it's not what matters” is something I'm willing to hear even though I usually make a different trade-off, I know a ton of C programmers who are fine with those languages not being memory-safe. But redefining the definition of memory safety[1] so that Zig can fit in, while at the same time arguing that memory safety is just a detail in the grand scheme of things, is just incomprehensible to me.
Also, your writing in this whole thread is full of resentment towards Rust and I don't think your animosity is helpful in any way: among the really tiny group of people who are currently experimenting with Zig there are people who actually love Rust and use it every day, I'm one of them and I know I'm not the only one. Bashing other language in a thread about a language you contribute to isn't the best way to be a welcoming community.
Good day.
[1] “Zig is (will be) memory safe”*
*as soon as you disable some compiler optimizations and use an allocator which is actually a memory-management runtime but not totally a GC because you need to free things yourself and if you make a mistake it will abort your program. Terms and conditions apply.
2. Nobody's goal is to use a memory-safe language. The goal is a safe and correct program. A memory-safe language is one way towards memory-safe programs, but the belief that this is the most effective way to achieve safe and correct programs is an interesting hypothesis, but it is just that. It is certainly possible that trading off those guarantees for something else -- like increased confidence across the board -- could result in safer, more correct programs, or at least not any worse.
> Also, your writing in this whole thread is full of resentment towards Rust and I don't think your animosity is helpful in any way
If you go over this thread again you will see that this is simply not true. I certainly did not bash Rust once. It is a very cool language with a bright future that many people love, but it just doesn't suit my personal tastes, that's all. And because both Rust and Zig are trying to provide safe low-level programming and they do it radically different ways comparing their approaches to safety is interesting. If my points are incomprehensible to you that might be because it is you who are acting with animosity and resentment that cloud your judgement.
Just so we're clear, I like Zig :-)
Of course not, but that doesn't mean Zig is less effective at achieving correctness. It just does so in a different way -- it trades sound guarantees for less sound ones and a simpler language with a faster cycle. Is one better than the other? Hard to say. Only empirical study could settle that.
Unlike C, Zig is (or will be) memory safe, although its safety can be turned off, and often is -- after testing. Unlike C, it provides powerful abstraction capabilities similar to those of C++. The fact that it can do all that yet be very simple seems to suggest at first glance that it's "like C" but that's because we've never had a language like that before. Zig's simplicity is misleading. It turns out you can do a lot with a very simple language. We knew that to be true for high-level languages like Scheme, but Zig shows it's possible in low-level languages, too.
I don't think this is actually true. The compiler is slow but not due to memory safety; the 'cargo check' command is rather quick and the compiler itself doesn't seem to spend a lot of time in the frontend, most of the time in spent in the backend, past the borrow checking phrase.
That's not to say you're safe to call other syscalls, many of them either require memory allocations in-kernel (see ENOMEM) or can block indefinitely.
There has been an interesting discussion about memory management in Ritchie’s PDP11 C compiler on the TUHS list this month https://minnie.tuhs.org/pipermail/tuhs/2020-August/thread.ht... from the era when large programs could not necessarily afford the overhead of malloc() so sometimes used sbrk() directly.
It'd be pretty unusual for junior or non-technical people to write code in "core" components of the system. Things like datafeed parsers, order handlers, inventory management, safety checks, networking libraries, exchange connections, etc.
But even with all these layers in place, you still need an actual strategy to run at the end of the day. Everything in the quoter can be optimized to hell, but if the strategy module is spinning for 1000+ microseconds because it's running some bloated ML model, then none of that really matters.
Usually the system engineers and the strategists are different people. Not always. Especially in the case of more straight-forward mechanical strategies. But anything reasonably complex usually requires dedicated quants with different skillsets than profiling C code.
From what I have heard, Optiver have a performance lab, which replicate real conditions with an exchange replayer and they can measure wire-to-wire latency for every release.
Hiring people for their maths chops as quants, you probably don't expect them to know about HW-level optimisations at the beginning of their finance careers, which, I guess, is the reason for such a performance lab. Build tools that help people bring their best skills to the table and catch regressions.
As well as having nicer syntax in general and real metaprogramming instead of the brain damage that is templates, zig promotes this kind of allocator-aware programming style in a way that's clean and idiomatic
In C++, you have to pass around an allocator to your templates. You can typedef this away if you want.
In Zig, you have to pass around an allocator as a function argument or a struct member. You can comptime this away if you want.
Is there some fundamental way that I missed that Zig changes this? If your actual complaint is that C++ templates are bad and you're saying Zig comptime is better, that's different than having woes about allocators.
(hint: there is no way to do this)
You don't need to do this if you're using allocators.
The other thing is that Zig doesn't seem to have any real plans to support C++-style closures right now. If they ever find a type-safe way to do it while supporting custom allocators, then that would be interesting, but at the moment I wouldn't say it's any better than C++ in this regard.
I actually have seen some C/C++ libraries that do allow changing the default allocator although it's usually only low-level libraries that bother to do this.
In general, I have not observed a consensus for 'idiomatic' C++, even within a single project. I say this as someone who wishes there was, because my job would be a lot easier if dependencies were less heterogeneous :)
I don't understand what's the difference.
I'm not an expert in writing real-time safe code, but I've spent close to a year working on allocation-free programming. Implicit copies of structs (value types) are as fast as implicit copies of integers, but implicit copies of types with owned heap memory invoke the copy constructor, which calls into the allocator. Or did I misunderstand your comment or get anything wrong?
With a handful of exceptions you essentially will never need to worry about this if you restrict assignment and argument passing of non-POD types to references within the critical code blocks. And in the case that you do need to worry about it, the copy constructor should be explicitly deleted anyway.
It's a footgun to be sure, but it's not a serious one with a bit of discipline. If you're used to doing real-time safe programming, you'll get paranoid about code that could invoke an allocator (or write your own).
It's a working strategy, but if you forget the `&` in `auto & x = document.foo; auto & y = x.field;` a single time, it might silently invoke the allocator. What actually happened to me was that it crashed because I returned a reference to a stack variable copied by mistake, when I meant to type a `&`. Pointers are probably less prone to accidental copying, but they have uglier dereference syntax and are nullable (excluding custom types).
Ever since that incident, I've been paranoid that I accidentally forgot the reference in another spot in the code. A few days ago, I debugged the code and set a breakpoint on malloc in the audio thread (LLDB crashes when listing threads, Visual Studio works) and found out my current codebase doesn't allocate on the audio thread. I hope I don't introduce any allocations.
To avoid this footgun, objects could be only copyable through an explicit `clone()` method like in Rust (which breaks std::vector<explicit_clone>), or by marking copy constructors as explicit (which you can't do to a std::vector).
Really though, you shouldn't have an owned STL instance like std::vector near your real time code to begin with. You're seeing one of the reasons people writing performant code don't use the STL at all, even if it has gotten up to par with handrolled solutions in certain benchmarks.
The irony.
That being said, a few notes on Rust on this project:
Cons:
- finding people proficient in Rust was a challenge (but that's why I go hired, so for me that was a plus;).
- in the first few years of the project, keeping up with language changes (before Rust 1.0 and even after because the project had been using nightly Rust until 2018) added.
Neural:
- the library ecosystem was nonexistent at the beginning, but because Rust has good C interop, the project just used C libraries for different things. Some where replaced by pure Rust ones later on, some didn't.
Pro: (Note: the project had important performance requirements (regarding CPU and memory consumption) so had Rust not been chosen, the project would have been written in C++.)
- When your Rust program compiles, it never crashes (except on an assert)
- I've spent exactly 0 minutes in a debugger on that project
- I've done massive refactoring without issues: you just fix the compiler errors (which are now extraordinarily clear!), you recompile and it works.
So overall, the Rust bet was a big success for this project! But you're right: the company wasn't a start-up and the company's ability to count on an existing team was vital here because hiring a new Rust dev would have been impossible in the first few years. With Rust becoming more and more popular each year, the hiring issue shouldn't be as acute right now (well, especially since Mozilla fired dozens of Rust-fluent developers earlier this month…)
With linear types you can guarantee that a destructor is run, so you can create objects with lifetimes that are not only bounded from below, but bounded by above. There are some common patterns in e.g. C libraries that rely on this--for example, you might register a callback on an object, and you want to assert that the object's lifetime is shorter than the callback's (so the callback is never called outside its lifetime).
Since Rust doesn't have linear types, you have to use unsafe{}.
This can be accomplished by a method that consumes the object and returns the closure (which now owns the object), and a closure method's that borrows the object back from it.
As in, you wouldn't write the code this way if you didn't have to. You do get memory safety in return... but you can see where the desire for a more eloquent approach might arise.
But you're not the first to think so!
That doesn't actually guarantee that the closure will outlive the object. You would need linear types in order to do that, and Rust does not have linear types.
In the particular API I'm working with, you pass the closure in to an object when you create it, and you have to make sure that the closure outlives the object. The only way to do that within the Rust type system is by making the closure 'static, which is... less than ideal. So you use unsafe{} instead.
This is because any object may outlive its specified lifetime. Lifetimes are only lower bounds. So if I have lifetimes 'b and 'a, and 'b : 'a, this only means that the LIFETIME 'b must be at least as long as the LIFETIME 'a, but any particular object with lifetime 'b may live arbitrarily long, as long as it lasts at least as long as 'b. And any object which is 'a may last arbitrarily long, but at least as long as 'a.
Per container allocators are on the roadmap : https://github.com/rust-lang/rfcs/blob/master/text/1398-kind...
Right now you could also go no_std (you still get the core library, which does not contain any allocating data structures) and use custom containers with a passed in allocator.
Zig is definitely a cool language, and it will be interesting if they can come up with good solutions to memory management!
But sentences like these in the documentation [2] would make me prefer Rust for most low level domains (for now):
> It is the Zig programmer's responsibility to ensure that a pointer is not accessed when the memory pointed to is no longer available. Note that a slice is a form of pointer, in that it references other memory. ...
> ... the documentation for the function should explain who "owns" the pointer
[1] https://github.com/rust-lang/wg-allocators
[2] https://ziglang.org/documentation/master/#toc-Lifetime-and-O...
While such use-after-free issues are not prevented at compile time, the plan is to ultimately have safe Zig catch them (and panic) at runtime, i.e. safe Zig shouldn't have undefined behaviour. Because this is done with runtime checks, the expectation is that those checks will be turned off (selectively, perhaps) in production, after testing has satisfied you. In that case, the guarantees aren't as strong as Rusts, but those guarantees come at a significant cost -- to language complexity and compilation time -- that can also have a negative effect on correctness. So while Zig's approach to safety/correctness is certainly very different from Rust's, I don't think it is necessarily weaker (perhaps it could even be stronger, but the question is hard to settle).
How would those guarantees have a negative effect on correctness? Are you thinking something like, you need to design your data structure / program in a non-intuitive way that makes it more difficult to get the logic right, even though you are protected from memory safety issues?
Aren't Rust's compilation time woes more due to the amount of IR the front/middle-end give to LLVM? I was under the impression that the type system-related stuff isn't that expensive.
What's not cheap, and is responsible for long compilation times (the order is arbitrary, the relative weights are highly dependent of the code-base):
- size of the code generation units (the whole crate vs individual files in C)
- procedural macros
- generics & traits
- interaction between generics & LLVM IR generation (a lot of bloat is created, to be removed by LLVM later)
- LLVM itself
Most of those are being worked on, but in the end it's mostly a social problem: as Rust users are used to long compile time, many of them don't especially take care of it, and most gains in the compiler are often outweighed by people writing slower code. It's already possible to write Rust code that compiles quickly, if you pay attention. The culture is evolving though, and more and more library authors are now mindful of compilation time of their crate (and the tooling to diagnose it is also improving).
Key takeaway: Memory safety isn't what makes Rust compile slowly, “zero-cost abstractions” is.
What one of Rust's designers told me when I asked him why they made the language so complicated is that nearly all of Rust's features exist to serve the borrow checker (except maybe macros). Once you have those features, and because Rust is a low-level language, you must have "zero-cost abstractions."
I don't know whether some other hypothetical low-language language could exist that gives you both sound compile-time memory safety guarantees as well as be a simple language that compiles quickly -- I would love to evaluate such a language, but we don't have one right now.
How?
By making heavy use of binary third party dependencies, every module gets its own binary library, no crazy use of metaprogramming, incremental compilation and linking.
My WinUI/UWP professional work compile in a fraction of my Gtk-rs toy applications.
I keep measuring improvements in this area, and hopefully Microsoft's own pain with Rust/WinRT might trigger some improvements.
I use Rust because I need performance, then I compile my Rust code for the exact CPU instructions available on my target machine and with PGO, binary dependencies can't do that.
Also, binary third party come with a lot of hassle (compiler version & options used can break your build) so I'm really glad Rust took the source-code dependency route instead (at least by default).
You can use binary dependencies though, as long as you compile everything with the same compiler it will works.
The only hassle is not wanting to learn how to use compiled languages properly, that is how we end up with the brain dead idea of header only libraries.
Regarding performance, Rust still needs to catch up with C++ in many domains.
There are plenty of reasons why C++ is my to go language outside my managed language options, despite my appreciation for Rust, and C++'s caveats of copy-paste compatibility with C.
If you want a full blown container you can use heapless, building a custom container is really straightforward and requires a minimal amount of unsafe.
In particular, it has two truly remarkable features that no other well-known low-level language -- C, C++, Ada, and Rust -- have or can ever have: lack of macros and lack of generics (and the associated concepts/typeclasses) [1]. These are very important features because they have a big impact on language complexity. Despite these features, Zig can do virtually everything those languages do with macros [2] and/or generics (including concepts/typeclasses), and with the same level of compile-time type safety and performance: their uses become natural applications of Zig's "superfeature" -- comptime.
Other languages -- like Nim, D, C++ and Rust also have a feature similar to Zig's comptime or are gradually getting there -- but what Zig noticed was that this simple feature makes several other complex and/or potentially harmful features redundant. Antoine de Saint-Exupery said that "perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away." I think that Zig, like Scheme -- and yes, there are others -- is close to that minimalist vision of perfection.
What a truly inspiring language. Rather than asking how we could make C++'s general philosophy work better as another increasingly famous language does, IMO, it asks how we could reshape low-level programming in a way that's a more radical break with the past. I think it's a better question to ask. Now all there's left to hope for is that Zig gets to 1.0 and gains some traction. I, for one, would love to find a suitable alternative to C++, and I believe Zig is the first language that could achieve that in a way that suits my particular taste.
[1]: I guess C has the second feature, but it loses both expressivity and performance because of it.
[2]: Without the less desirable things people can do with macros.
I'm curious where this impression of Zig comes from, as this is precisely what Nim has set out to do: a small core extensible via metaprogramming. Are there features that Nim implements which go against this premise? if so, what are they? :)
Nim has generics (plus concepts), templates, and macros. Zig has just comptime, through which it achieves all those goals (minus some macro capabilities that it deems harmful anyway) with just one, very simple, cohesive construct. You could argue on whether you like this or not, but you can't argue that Zig's approach isn't fundamentally more minimal. Zig is a language you can reasonably fully learn in one day; I don't think you could say the same about Nim.
---
Also, note another remarkable feature. One could define a language called Zig' with the following properties:
1. Every well-formed Zig program is a well-formed Zig' program (i.e. Zig' accepts all Zig programs, potentially more).
2. Every Zig program has the same semantics as the identical program when interpreted in Zig'.
This means that to analyse the semantics of a Zig program you can pretend it's a Zig' program; in fact, you don't need to create an interpreter for Zig', you can just pretend it exists. Why would you want to do that? Because Zig' is simpler. How? Here's the kicker: Zig' ignores comptime completely. It is a very simple, optionally-typed, dynamic language with reflection.
In other words, to analyse the semantics of a Zig program you can forget about comptime and pretend it can do everything at runtime (and treat comptime as a pure semantics-preserving optimisation).
This is not true for languages with macros, as they are not "erasable".
"some macros" are downplaying Nim macros, it's like saying Lisp has some AST rewrite capabilities.
Nim macros goals are two-folds:
1. adding functionality to the language without baking it in the compiler. A prime example is ``async``, ``async`` including a nice async/await syntax can be completely implemented as a library without reserving keywords to do things like `pub async fn`.
2. Automating away boilerplate.
From what I understood, Zig comptime is only about making compile-time function evaluation first-class.
In other words, there is a capability here that Nim really, really wants, and that Zig really, really doesn't want, so on that front they are not competing in their designs.
I also take issue with your statement that zig is "more minimal" since that only applies to the user's perspective -- from the compiler's perspective, macros make a language far more minimal. But i vaguely recall already discussing this distinction with you so i don't want to rehash it.
At any rate i will definitely be paying attention to andrew's progress, he has a really clear vision.
Macros are controversial. I love them in Scheme and Clojure, but I wouldn't want them in any language aimed at a larger, more mainstream crowd. At the very least, macros introduce another meta-language to know (and if they're in a language with a complex type-level language like Rust or Haskell then they're a third language within the language), but I think it's a matter of personal aesthetic preference.
> But i vaguely recall already discussing this distinction with you so i don't want to rehash it.
Maybe we did. :)
See my comment here about "Zig' ": https://news.ycombinator.com/item?id=24293611
Perhaps now you see what I meant when I said that Zig's simplicity hides its radical design.
You can also do `const a = static(foo(x, y, z))` to force normal function to be evaluated at compile-time and store them in a constants.
Hence you don't need to use macros for compile-time evaluation in Nim just like in Zig. However macros are necessary for AST manipulation.
It's indeed an inspiring language, and rust is taking inspiration from it already: https://github.com/jswrenn/project-safe-transmute/blob/rfc/r...
> lack of generics
I can't wait before Zig2 comes and eventually adds generics…
Edit: clearly gp was not mistaken, just imprecise.
But C++/Rust can never have Zig's primary feature -- simplicity. Zig's power is not that it has comptime, but that it has little else.
> I can't wait before Zig2 comes and eventually adds generics…
No need. Zig gives you the same capabilities as generics do, only through a separate feature that other languages also have in addition to generics. In other words, it has generics, but without having generics as a special construct. Zig recognises that once you have that other feature (compile-time introspection) you don't need generics as a separate construct, but they can be just an instance of that single construct.
Sounds like a Go pitch, except Zig ain't Go. And while comptime is a cool feature, it's also a really complex one!
> In other words, it has generics, but without having generics as a special construct. Zig recognises that once you have that other feature (compile-time introspection) you don't need generics as a separate construct, but they can be just an instance of that single construct.
This has advantages (only one feature to know), but it also has a big drawback: the lack of orthogonality. C is also simple, for instance it has no concept of errors (only return values) or arrays (only pointers), but most people won't consider this a good idea (and Zig didn't follow C on either of those two design points)
Zig is cool, but I hoped the “generics are too complex of a feature” meme would die now that Go is getting generics, and I'd be really sad to see come back…
No, it's a very simple one, so much so that it's erasable: https://news.ycombinator.com/item?id=24293611 And still it is probably the most complex aspect of Zig.
> Zig is cool, but I hoped the “generics are too complex of a feature” meme would die now that Go is getting generics, and I'd be really sad to see come back…
You've misunderstood me. Generics are a good thing -- if that's all you have. But if you have generics and procedural macros, it turns out that you can do the work of both with a feature that's simpler than either. The capability generics add is a very important one, but given that low-level languages need another one as well, it turns out that generics can be subsumed into that one without being a separate and additional construct. Zig has generic types and concepts/typeclasses; these just aren't atomic language constructs.
Here I think we just have a different subjective perception of what simplicity is. I much prefer have two orthogonal systems which do their own business than having a single more powerful tool than do both (like having slices + references instead of the all powerful pointer)
Anecdotal note: more than 10 years ago, the Go team pitched why they didn't need generics nor macros, because code generation would solve both problems (+ others), and now they're on their way back to add generics to Go (with a lot of hassle).
OK, but that's not quite the situation. Here we're talking about languages that have, or will have, the single "more powerful" construct, and also the more specific, special case one, as two separate constructs, even though one of them would have sufficed.
Again, Zig has parameterized types, and very elegant and powerful ones -- they're functions that take some types as argument and return a type. It just doesn't have generics as a separate construct. Rather, it is a special case of a more general one (that Rust and C++ will also have).
That is a confusing part with C. C do have a concept of arrays, but not as function arguments. You notice it first with multidimensional arrays.
https://ziglang.org/documentation/master/#Introducing-the-Co...
I mean, the workarounds are horrible code generation tools and reflection. How were those ever not considered to be more complex than generics?
And if you see the flack that go gets for not including generics then I'm not so sure that that is a great way to get people to adopt your language.
That being said, Zig's comptime is not a proper replacement for typeclasses/traits. Zig can do both comptime duck typing and vtable-based dispatch, but it cannot do proper bounded polymorphism type checking. It always fully evaluates types before type checking them. This might make it difficult (or impossible?) to provide type checking error messages of similar quality to Rust. I'm not sure if there are any other practical consequences for realistic programs, though. I suspect there might be issues around interface stability gaurantees, though I can't quite put my finger on why.
Having said that, I do support a proposal for specifying a type at the parameter declaration with some type -> bool function.
I like this quote by Leslie Lamport about comparing formalisms (he talks about specification languages rather than programming languages, but the sentiment is the same):
> Comparisons between radically different formalisms tend to cause a great deal of confusion. Proponents of formalism A often claim that formalism B is inadequate because concepts that are fundamental to specifications written with A cannot be expressed with B. Such arguments are misleading. The purpose of a formalism is not to express specifications written in another formalism, but to specify some aspects of some class of computer systems. Specifications of the same system written with two different formalisms are likely to be formally incomparable… Arguments that compare formalisms directly, without considering how those formalisms are used to specify actual systems, are useless.
Particularly great is Zig's handling of both macros and generic types, the answer to both of which seems to be: just evaluate them at compile-time with regular functions, no special DSL or extra syntax. Andrew mentions in the video a big drawback of this system - implications for IDE complexity and performance. I imagine the performance side of this could be (maybe is?) mitigated by limiting recursion depth/loop counts for compile-time work.
I'm not particularly interested in taking on a language with manual memory management and the responsibilities it entails, but I would love to have access to Zig's compile-time capabilities, if it were available with some more memory safety.
This sounds like "we can't guarantee the most important thing, so it's unclear whether it's useful to guarantee this other thing," but that's a bizarre statement, so am I misinterpreting?
For memory related issues I find it sufficient.
One aspect where it is probably not as good as Rust is for threading related issues, as it relies on inserting runtime checks which may or may not trigger depending on the number of threads attempting access.
It also works with iterators-generally by sticking some extra data in the iterator in dev builds, so it can check for out of bounds access.
If I do have some code that uses C pointers + size, I'll insert some dev build assertions.
https://github.com/ziglang/zig/pull/5998
edit: For more details, see
We see similar tradeoffs of soundness in formal verification as well. We're not talking about exactly the same thing here (because affine type safety is compositional) but the general principle is the same: soundness has a cost, and it is not necessarily the most efficient way of achieving a required level of correctness.
Anyway, I think that both Rust and Zig have very interesting approaches to safety, but I don't think we know enough to claim one is more effective than the other at this time.
RAII is not precluded by explicitness, you could require all values that require cleanup to be syntactically marked in some way and it would still be RAII. defer also cannot handle resources whose lifetimes do not correspond to nested scopes (eg. the elements of an ArrayList) like RAII or a GC can.
I think this one is TBD.
> defer also cannot handle resources whose lifetimes do not correspond to nested scopes (eg. the elements of an ArrayList) like RAII or a GC can.
Yep, defer won't work if there is no known lifetime scope, but I think this one is actually a good tradeoff to make in a low-level language. Don't get me wrong -- I love tracing GCs and think that they're the right choice for the vast majority of application software, plus there have been great strides made in GC capabilities in the past few years, but in the domains where low-level languages are appropriate there is a different set of constraints. Low-level programming is not like high-level programming, and IMO it's wrong to even try to make them look alike.
It can, it's just more explicit about it. In a language with destructors, you'd do RAII here by having a list destructor that cleans up each element in turn. In a language with defer, the same destructor becomes a regular function that you'd invoke in the deferred expression.
The other issue is since there is no generic notion of a destructor, it isn't possible to write generic functions that destroy elements. If you call, say, replace_range on a list of strings, it will leak the replaced strings.
That's separate from the destructor for the entire list. It does mean that the code that removes an element from the list has to explicitly invoke the destructor for it - which is in agreement with using "defer" to explicitly invoking destructors for locals.
> The other issue is since there is no generic notion of a destructor, it isn't possible to write generic functions that destroy elements.
But you can have a generic notion of a destructor - that's orthogonal to whether destructors are invoked explicitly. You just have an interface (or trait, or whatever it's called) that exposes a destructor method for a type.
My experience is that Rust's approach is definitely better in terms of correctness than using valgrind during testing.
My intuition is that the advantages Zig brings to the table will not tip the balance.
That being said, the choice Zig makes is absolutely the right one. Rust fills the niche of a better and more correct C++ without fixing the issues of slow compilation and language complexity.
Zig fixes so much of what's wrong with C without abandoning the advantages of language simplicity and locality of reasoning. I love Zig, but need a medium sized non-work project for it.
- It's a very low level language. Having written mostly Python for the past few years, it is quite the contrast. I had to force myself to think in C to get the train going
- Getting my head around the error handling took more time than I'm willing to admit. In the end, it's like having exception but being more explicit about it. It feels nice when you get the hang of it
- The documentation of the standard library is severly lacking, to be fair the language is still very young. More worrysome, it feels very cluncky.
- No proper string support. It is sad that a modern language still goes down that route after Python has shown that correcting this is both definitely worthwhile and a word of pain.
- I have the feeling that optional and error union types are a bit redundant, but I have not written enough Zig to have a real intuition on that. Maybe it is just that I understand monads now.
Optional and ErrorUnion are a tiny bit redundant in that one could represent the Optional as another value in an ErrorUnion, and that might even happen as an optimization step in the case of ?!/!? types at some point in the future, but they have very different handling in the language as they are used for very different things.
I personally like that Zig doesn't bother with "strings" at a language level at all and just considers everything as arrays of bytes. String handling is a complexity nightmare and I feel that Zig wisely chooses to be simple instead.
I have not expressed myself clearly: the auto-generated documentation is severly lacking. The API of the standard library is clunky. To be fair, both those points are getting better. And yes, the language is very young and I understand that there are more pressing issues with the core language itself.
> I personally like that Zig doesn't bother with "strings" at a language level at all and just considers everything as arrays of bytes. String handling is a complexity nightmare and I feel that Zig wisely chooses to be simple instead.
It is definitely simpler, alas not everything is ASCII and arguing it should be to make life easy for programers is hardly a reasonable stance.
Also, maybe it is not clear in my comments but I actually enjoy Zig.
Now you may argue that this can be handled nicely in the standard library without changing the language. This is correct, but there will be some frictions with string litterals.
The Rustaceans should be taking notes on this.
Ans also, that's the strength of C and its simple ABI.
After that you should already be in great shape and you can read the standard library for examples of useful patterns.
Semantic Analysis [533/803] ./main.zig:4:14: error: container 'std.debug' has no member called 'print' std.debug.print("Hello, {}!\n", .{"World"});
But in general I have gone for for generalist not specialist in my career.
But there is diminishing value in how deep you want to go into the rabbit hole. Of course there's always more to learn, but with C you're fairly quickly leaving the language and move into the layers of compiler- and hardware-trivia (good to know nonetheless, but often not really relevant for being productive in C) where in other higher-level languages you're still working your way through the standard library ;)
It's not as big of a problem these days with things becoming less heterogeneous; almost everything is little endian now, much of it 64-bit but at least 32-bit, and we can kind of rely on POSIX being there most of the time. Most new code uses stdint.h and is explicit about word lengths by using int32_t, etc. and follows good conventions there.
But venture off the beaten path into odd microcontrollers or into retro machines or port older code or whatever ... and there's glass hidden in the grass all over.
C also exposes a model of the machine that looks low level but behind the scenes a modern processor does all sorts of branch prediction and pipelining and so on that can blow up your assumptions.
What looks like optimized clever C code can actually end up running really slow on a modern machine, and vice versa.
The good thing is that mastering one of these languages gives you conceptual tools which help with becoming at least competent in the others, if not mastering them as well.
I can't help but feel like in our industry C is successful (vs its 80s competition of Pascal/Modula-2, or Ada etc.) partially because of some of the same reasons that Git is successful now. Yes, it is powerful and flexible; but also in some ways unnecessarily arcane and 'dangerous' and _this gives the user a feeling of cleverness_ that is seductive to software engineers.
Put another way: Most of us enjoy the mental stimulation of programming, and we enjoy the mental challenges (in general). C makes us feel clever. Witness the "obfuscated C programming contest" etc.
Same thing that has led to nonsense 'brain teaser' whiteboard-algorithm tests at job interviews. IMHO it's in many cases for the benefit of the interviewer's ego, not the company or the interviewee ("gotcha! no job for you!").
</>
Yep, only C makes me feel stupid (but I enjoy that experience too!).
Luckily my day-job has nothing to do with mental gymnastics even though I'm a software engineer at Google and work in plenty of low-level stuff. Most sensible software development bears little resemblance to the stuff on whiteboards in coding interviews etc.
After 20 years of this I know the right thing is to reach for a library, and if that doesn't exist, then reach for Knuth or some other reference rather than try to write it myself from scratch.
I much prefer writing Python or Lisp code than C++, but I can't do my job in Python or Lisp code, so I write C++.
Don't think Zig is worth it when it doesn't have linear or affine types.
I think that our community's equivalent of "where's my flying car?" is "where's my higher-level language?"
Ηelicopters are flying cars and they are everywhere for you to use. But some people prefer to use a bicycle to commute to work, rather than an helicopter. I'd even say that most people would prefer to take a bicycle every day than an helicopter.
The same thing with lower level languages. Sometimes you do not want to be burdened by the limitations of a "high-level" language.
I see very few people suffering from such burdens, but a great many suffering from its exact opposite: using a low- or mid-level language to write hundreds of lines where ten lines in a higher-level language would suffice and be more easily verified as correct.
I see a huge load of people suffering from those burdens. Higher-level languages tend to be less efficient and less optimal. Yes, they take a burden from the programmer and move that burden onto the end user.
So the programmer is having the easy life, while every user now waits a second longer for the program startup, a second longer for opening the file dialog, and so on. Doesn't sound much, but if you think about it: Worst case is: 1 programmer shaved off two weeks of work on an app that is used by every person on the planet. 7 billion users lost 1 second. In total, humanity lost roughly 200 years of productive time when every person starts the app exactly once.
Modern computers are incredibly fast and we as programmers use that brutal power to be more lazy than before instead of leveraging that power to all of the users of our tools. We could have systems that go from clicking the power button to being readily available for use in less than seconds. Think about this when you chose a high level language that exchanges programmer convenience for runtime cost and think about if it other peoples time is worth your lazyness.
</rant>
disclaimer: don't take it personally, i'm just frustrated about imperformant software
I've spent much of my career writing low-level code in low-level languages because I had to, usually in C and usually because I was in resource-constrained environments where tight control over CPU and memory footprints was necessary. There's absolutely room for languages that improve programmers' lives in that kind of environment while remaining suitable to that purpose. I'd put Zig in that category, and I find much to admire in it.
However, outside that domain, once you have even a little freedom from those constraints, it makes no sense to use a language designed around them. When even something as simple as manipulating a few strings or updating objects in a map/hash/dictionary requires careful attention to avoid memory leaks or excessive copying, and your code is doing those things a lot, you're using the wrong language. A language that "protects" you by guiding you toward adding the right boilerplate in the right places honestly isn't much of a help. Most code should be written in a truly higher level language, where things like circular references don't require much discussion except by the language implementors. The problem of how to do that without going full-GC and having to deal with pauses is where people should focus their attention, not more languages that just change which ceremony you must adhere to.
Low-level languages encourage the use of clean ownership patterns, which ultimately leads to cleaner design.
Maybe the authors of that code just had different priorities, such as maximising compilation time.
I work in a multi-million-line codebase, a significant majority of which is very far from that "need perfect control" domain but is written in a what I'd call a mid-level language - a high-abstraction dialect of C++. So I'd say GP is correct, that too many people are stuck writing code in the wrong language for the task at hand. The need for languages like Zig to improve the lower-level experience (which, as you say, is not going away) and the need for higher-level languages for more common types of programs are not in conflict. They're complementary. It's the mid-level languages that need to DIAF, because they're not really suited for either and pretending to be more general than they are only encourages people to make choices that hurt them.
Burroughs, Mesa, Modula, Oberon, Interlisp-D,... were on the right path, but in technology not always the best ideas win.
Give it a few years and everybody is going to benefit from better implemented "lower-level" applications, thanks to Zig and other languages attempting to do the same.
I think Zig has the potential to be the best of them, fwiw.