Zig's New Relationship with LLVM
kristoff.it
kristoff.it
It manages to address all of C++'s biggest shortcomings, which, in my view, are 1. language complexity, 2. compilation time, 3. safety -- in this order -- and it does so in a language that is arguably simpler than C, can be fully learned in a day or two (although the full implications of the design might take longer to sink in), and also inspires a new approach to partial evaluation, replacing generics and value templates, concepts/traits/typeclasses, constexprs/procedural macros, macros (or, at least, the "good parts" of macros) and conditional compilation with a single, simple feature.
But what for a C programmer (systems/embedded) remains to be seen, is, if any of the new languages (say for example Rust, Zig, Odin) can (or want to) offer one of C's strengths most language designers do not think of (at least it looks to me like that). And this would be for me "leave the language alone (for the most part)".
If i may quote from Zig's website (https://ziglang.org/#Small-simple-language): "Focus on debugging your application rather than debugging your programming language knowledge."
I have yet so see a community driven language that fulfills this promise in a comparable way to C's. There's always "this little feature" necessary, or "this cruft to clean up". You name it i've seen it.
If you want people like me (and trust me HN community, there's a GIANT silent majority out there) to use Zig e.a. for production in the embedded and systems space where product lifecycles are counted in decades, you will have to provide this feature. Sure, you can call me a dinosaur (still under 40 :-)), or a BLUB programmer or what not. But language designers wanting to (slowly) put C to rest, should, imho, not forget about this "feature".
In this space you really want to focus on the real problems, not on problems of keeping up with the latest and greatest.
Side note for all the "if all you know is a hammer" responses: Please don't compare programming languages with tools. This is a bad analogy. Programming languages are more like materials. Once chosen and the product (the source code) built, you cannot simply change it without starting from scratch and a new product (new source code).
So, downvotes, here i come ;-).
C is not going any where.. we all know that. But it is not the best tool for accomplishing many systems programming tasks. IMO, this has probably been true since Ada95. Ada2012/SPARK, Rust, and probably Zig make this even more clear.
For some domains, C was the ONLY tool available... e.g. C is/was the only language with board-level support and an available compiler.
At my place of work, we presently use C for embedded code because it is best supported for the microprocessor we are using... but we'd rather use Rust, Ada, or Zig. we may explore using Nim to generate C with static-only allocation.
Rust has the (imo) above stated problem. Also, if you avoid dynamic allocation, Rust's advantages are not as compelling in the real world as advertised (personal opinion). They are there, but IMO minor.
The problem right now is that we have a chicken and egg problem. You said you work in embedded too, so you know MCU bugs exist. Use anything but the "recommended" tool chain, and the compilers will be blamed, no matter how much proof you have that it is the MCU.
Don't get me wrong, i'm definitely keeping an eye on Zig. There is lots to be liked. I cited that its goal is keeping the language simple. But only time will tell if the developers can keep up to this promise. I, personally, don't trust languages developed by communities in this regard. Reality has (sadly) proven me right.
(Are you talking about ide licences, the commercially supported version of the compiler, ramp-up time to productiveness, or maybe something else?)
Somehow I've become interested in the economics of Ada, and am considering it in comparison to several other strongly typed systems programming languages for a learning project. So I'm a bit curious.
Alas... not to mention debugging tools, etc. That's why I've gone down the Nim path at work as well, and it's actually pretty fun once you get the basics. Having JSON support in a few kB's of flash is nice. I've ported enough of the Nim standard library to FreeRTOS/LwIP to be able to use it with the ESP32! If you want to take a look and see how it integrates with FreeRTOS [1].
I'd recommend trying out Nim's new ARC garbage collector if you have more than a few dozen kB's of RAM. The ARC GC is pretty friendly to working with C based-boards as it can use the native malloc and any native debugger mechanisms. The references are non-locking to it's pretty low overhead, but if you do try it run `-d:nimArcDebug -d:traceArc` first to see the overhead. You can't use `async` without cycle collection (ORC) however.
Of course no-GC works too and you can annotate functions to guarantee no GC (I think). Just less fun if you have the RAM available.
1: https://github.com/elcritch/nesper (Warning: WIP)
It's currently much more expensive to port an Ada, Rust, or even C++ toolchain to a new platform, so I'd expect C to continue to be dominant on lower-spec chips. Here's a previous comment adding a bit more technical detail to that argument: https://news.ycombinator.com/item?id=22822931
I have to say, zig really captures this feel. The only places where it gets a tiny bit strange in the way you complain about is comptime, but imo it is a very good trade-off for ditching the preprocessor and ditching make. And anyways comptime is far more easy to reason about than some of the mangled ways in which people who use C (in prod) use preprocessor: It took me weeks to understand the ffi header file for the erlang VM, for example. Three languages for the price of 1.25. not bad.
As an example of this one of the features they added was NLL (non-lexical lifetimes) which was designed to help iron out some of the annoying places the compiler couldn't see what you were doing was safe. One of the constraints they gave in implementing it was no changes to existing code that compiled, the way they did it as a non-breaking change was to ensure that the new system only allowed programs to compile that didn't compile before.
It sounds like you aren't interested in Rust for other reasons but wanted to point out that language hears your concerns about breaking changes being the worst.
Also, editions are no response to the issue of new features being added to the language, changing what is considered idiomatic, okay, etc. If Rust turns out to evolve constantly like C++ has in the past, then it'll be hard to keep up with it, in addition to the great one time cost of learning it in the first place.
Basically the idea was rather than making it as easy as possible to stay on the oldest version they focused that effort on making it as easy as possible to update once you decided that was what you wanted to do.
This one is tricky for a few reasons that don't apply to most things.
Especially after NLL landed on 2015 ><
If I am inspecting code for safety that is doing low level things then I want to have a clear and simple (imperative) model in my head. I don't want to spend the review admiring how clever the author is. I want to know as directly as possible what is going on in the machine.
In another domain the ability for the author to do clever things could make me very productive but it doesn't matter because the safety of low level code is not a concern.
C++'s `unique_ptr` is a "newfangled feature" but is not the latter (ignoring custom delete functionality).
I tried to account for the discrediting of zero cost abstractions (which I buy) but I think you are adding the latter onto that which I didn't get from the person I was responding to.
> I want to know as directly as possible what is going on in the machine
Certainly not in multi-threaded code on an x64 machine (and IIRC ARM is trending in that way as well). There are too many abstractions to actually do that, the best you can do is understand the abstract machine of the language and the rough edges translating that into real machine code entails.
> safety of low level code is not a concern
I will remind you that in the case of Rust safety is where all of this comes from not productivity.
Writing a mutex by hand is hard and you will make mistakes, here is a library construct to do it for you. It will be more complex than what you would write but it will be safer because others have worked with it. etc. etc.
When working with minimal computation resources using "works for everyone" constructs stops being zero cost and starts being expensive both conceptually (what were the poisoning rules again?) and performance wise when the cost of providing your abstraction is hard (why can't I arena allocate an RC?)
Those are the situations where Rust and even C++ have difficulty shining and C wins out. It provides so little you never have to pay for anything but what you absolutely need. But remember the things you lose out on are from every category you can think of, not just one. Performance, reliability and safety are all often sacrificed in some amount when writing custom code.
When you write code instead of a random Joe in a garage you know who to blame when it blows up. When you write code instead of using the larger community code you give up the often great characteristics in every metric for your more fine tuned characteristics in some metric generally.
Sorry that got wordy... TL;DR nothing about most low level languages up to things like Java and C# IMO stops you from having a simple imperative model in your head so using that to chose C over something else is missing the point.
On the other hand if you change the language, say every two years, when is the right point to dive in? Is it at some point (say i missed the last four years of language development) hopeless to catch up?
I mean, this is only true if you ignore the myriad ways in which C allows you to manifest memory unsafety and undefined behaviour
"I mean, this is only true if you ignore the myriad ways in which C allows you to manifest memory unsafety and undefined behaviour"
"Yes, but in Debug/ReleaseSafe it's much stronger than C with sanitisers."
Are you sure you hit all the execution paths during development?
Pervasive casts? Not under MISRA C.
Look, I'm not criticising Zig on a technical level (I mentioned above I'm keeping an eye on it because it looks interesting), but I don't have any trust in community driven languages keeping it simple (although Zig promises to do so, see link above).
EDIT: Corrected formatting.
No, and, in fact, I specifically said that C has unparalleled tooling in the formal verification space: https://news.ycombinator.com/item?id=24618856
But, 1. as much as I like formal verification and am actively involved in the space, we're talking less than 1% of C code that uses those tools (unfortunately), and 2. because Zig is so simple -- simpler even than C from a formal analysis perspective because of slices and casts -- if it ever becomes successful, I have no doubt we'll see such tools emerge.
Of course, the other 99% can't use Zig either because it's not even "out" yet. Obviously, a product that doesn't even exist yet can't replace any product that does, and no one is suggesting that Zig -- in it's current virtually nonexistent form -- is a replacement for anything, let alone for C. We're talking about what Zig could do when it exists in some production-ready form.
I think Rust is a great, worthy language, and for a lot of use-cases it makes sense to optimize for trying to just put everything on the stack, but there are a lot of other cases where what you really want to do is own allocation by yourself instead of trusting it to the compiler. I haven't tried Zig, but I really want to see if it can deliver on this.
The new test allocator checks your code for memory leaks, use after free, and double free in a highly ergonomic fashion. I think is fantastic that zig encourages writing tests, and I think if you get full test coverage in your zig code with the test allocator, you will probably solve 99% of your memory errors.
I know this is possible in C++ (see alexandrescu's allocators talk from CppCon 2017?) But I would be surprised if anyone does it.
With zig, not 100% comparable (like not sure about gdb, but zig stacktraces and error traces are fantastic) it's all done in one language. Such a lowering of mental overhead and entry barrier!
The only thing you have to learn is that at comptime you get access to some magic (like the ++ operator)
The only hard part is figuring out how to pass -fsanitize=address into your compiler via your complex build system! And building with symbols.
That is inherent to the C/C++ toolchain and not a problem that sanitizers can address.
A lot of the advantage which Zig has over C is exactly in not having to support a towering edifice of tools and hacks which is older than I am. This is true to a significant degree of Rust and C++ as well.
This could be done in Rust. There is, for example, the simple bump allocator bumpalo [1].
It would be nice if the the std collections supported this (in planning, but hasn't seen much progress), and most dependencies would not be built around a manually passed allocator.
> Rust assumes most of the time you'll be passing around references to values or small structures allocated on the stack.
Can you clarify what you mean here? All of the std collections (Vec, HashMap, etc) use allocation. There are also `Box`, `Rc` and `Arc`, which allocate and are used everywhere.
Again, I do not think this a weakness of Rust, and I do not think it should change. In very many cases, the Rust approach is very helpful.
The point is there are also cases where imperative, explicit memory management is desirable.
I agree that is the case, and Zigs convention of passing an allocator around and making those calls very explicit has merit.
I think you are over-estimating the amount of "magic" in Rust though.
The allocation-related logic is not part of the language or the compiler, but comes from the std (or alloc) crate. You just don't usually see a "malloc" call because it is hidden behind types in the standard library, which default to a global allocator. You still "manually" allocate by calling eg `Box::new()`.
The only magic is that Rust calls a destructor ( Drop::drop) when a value goes out of scope. Types can implement drop and use it to deallocate or decrement a reference counter. This is made possible by using an affine type system with lifetimes and move semantics.
This is conceptually not that different from doing something like `defer allocator.free(x)` in Zig, though less explicit.
ps: I realize that conventions make a difference. But nothing in Zig would stop me from having a global variable with an allocator, and using that without passing it around.
That's correct and you probably should do that in your own applications, but it will be frowned upon in libraries, and people will not choose to use them if you make that choice.
There's also another pattern, where you stuff a pointer to your allocator in your struct and have your "object" carry around its allocation system, you don't have to pass around allocators that way.
And it doesn’t have to be just one allocator either, you can easily use multiple allocators simultaneously.
Can you expand what you mean here?
I really like Zig, and there is definitely a big design space to explore in creating a modern low-level language that doesn't come with the complexity of something like Ada/Spark or Rust.
This binary patching + daemonized compiler approach is particularly exciting.
But as a language, I'm not aware of any features that would amount to a "radical breakthrough". As far as I am aware, it is a modernized, low level language, with nice compile time evaluation, async, and some runtime-provided safety guarantees. But nothing that is so novel from a type system / language design point of view.
Happy to be corrected though!
I'm not sure I like the "better C" label. After all, Zig allows you to easily do stuff that's virtually impossible in C, and only possible in C++ with templates and constexprs and concepts. It also doesn't have pointer arithmetic and pervasive wild casts, so it's not a "syntax-sugared Assembly". Its only similarity to C is that it is a low-level language and that, unlike other low-level languages that aren't C -- like C++, Rust, and Ada -- not only is it not among the most complex languages in the history of software (all three of those have probably secured their places in the top-five), but it is probably among the simplest.
> I'm not aware of any features that would amount to a "radical breakthrough". As far as I am aware, it is a modernized C, with nice compile time evaluation, async, and some runtime-provided safety guarantees. But nothing that is novel from a type system / language design point of view.
AFAIK, general partial evaluation with introspection as a single mechanism to do the work of generics, typeclasses/traits/concepts, value templates, macros, and conditional compilation -- combined with a general error reporting mechanism that is shared between runtime and compile-time -- has never been attempted before. It is revolutionary.
I agree that Zig is more than a "better C", and I edited my comment.
Regarding `comptime`: I haven't fully grokked it, but it looks very similar to the capabilities of D (and in part Nim). Do you by chance know those languages and can give a comparison?
(Languages like Idris also allow to express a lot at compile time, but that's quite a different domain)
(To be clear, I don't use Zig, yet.)
Have you seen http://terralang.org/
BTW, even Zig's build language is just Zig!
Fully optimized release builds will surely still have an option to use LLVM
So, zig is new to me but looking at an overview of the feature in question here this feels pretty hyperbolic? I certainly agree that the approach it takes is syntactically less noisy than angle-bracket generics/templates (which I dislike quite a lot) but it doesn't seem particularly revolutionary vs. the way evaluation of types in haskell works, for eg. A function that produces a type based on type arguments is roughly a typeclass.
And the intermediate products of a function with comptime arguments are still pretty much semantically equivalent to a C++ template anyways, just with the arguments in a different place.
It is cool, and I like the angle it takes, but it seems more synthesis than revolution to me. Am I missing something?
I guess I should also mention my inspiration, the C++ dyno library (https://github.com/ldionne/dyno)
I don't really see how this is relevant to typeclasses though ^^
All that said, I’m personally okay with the decision to omit typeclasses since the language is meant to compete with C, not C++.
I'm sure there are a good handful of other fields that currently write "minimal C-like C++" and I think Zig competes with C++ in those cases.
I don't know about that. "that of C's "syntax-sugared Assembly" is pretty much how most people want to do system programming. Don't systems programmers want to be as close to assembly without delving into the pain that is assembly? Also, I don't know how "ground up" it is. Seems like zig is just more syntactic sugar on top of C to me.
Wasn't there a big push for D to supplant C/C++ a while back. Now Rust has joined the fray. Why would Zig succeed when D failed and Rust is failing? What is it that Zig offers that D or Rust doesn't? And does that justify the migration of legions of C programmers and mountains of C code to Zig?
I'd never heard of zig until now, but as a community having top level "how to get started" website using features only available from the Master branch of github is... not a good way to get people into your language. It'd be nice if your getting started guide only used features from the tagged latest version at least... (I get that it's moving fast, but geez this is a Hello, World sample that fails)
edit: to clarify I am the maintainer/owner of ziglearn
With a line of text that I must use the latest dev/git code (etc.), my choice would have then gone the other direction to just download a binary. The number of websites which "just download a binary" when a suitable package is already available in the repo are a very high number (for Arch), so this thought process is standard for a guy like me.
I think yours is a fair point, but that's how things are. If you prefer using a tagged version and not have to be too involved in the latest developments of Zig, please consider waiting until 0.8.0 at the very least.
Don't get me wrong, it should be clarified on the ziglearn website anyway.
"Literally the first thing" that I see on the download page is:
>Releases
>You can also [install Zig from a package manager.](https://github.com/ziglang/zig/wiki/Install-Zig-from-a-Packa...)
Going by the AUR package script and the table in the wiki itself, those instructions will install 0.6.0. This is also what gravitas said in the OP of this thread.
Anyway, I see ziglearn.org has now been fixed to make it clear, with https://github.com/Sobeston/ziglearn/commit/10b14426e26ffcdb... , so all's well now.
Imho it would be unreasonable to even try keeping backwards compatibility in a language that tries to find the global maximum
> but geez this is a Hello, World sample that fails This is because the 0.7.0 release cycle changed a lot about logging: std.debug.warn was used as a general purpose log function in 0.6, but is now deprecated to move people to use the std.log namespace now
> It'd be nice if your getting started guide only used features from the tagged latest version at least...
The problem here is that package manager maintainers only package tagged versions which isn't really a good thing for zig. Your version is kinda outdated as soon as you downloaded it. Most projects won't break anymore on a daily basis, but there is still changes, bug fixes and additions.
I'm currently waiting for some PRs to be merged, so i can continue my own projects. You won't get this when sticking to tagged releases (and it will make updating your projects way more painful than keeping them up-to-date on a daily basis)
But yeah, ziglearn could note that using a tagged version has these problems and that zig is moving so fast that sticking to a tagged version will yield outdated code very quickly
What does this mean? I can't think of any way to interpret "find the global maximum" that applies to Zig but not other languages.
Please, please - make a Hello, World which is not so intrinsically tied to random features on Master to just work across all the iterations. Hello, World is supposed to be the simplest, non-breaking easy to compile no esoteric compiler feature thing in any programming language in the world.
For example the current `anytype` type has replaced `var` and running zig fmt on a project I hadn't updated yet changed all references.
That said, Hello World, when taken seriously, it's not that simple of a program and I'm working on a talk for Zig SHOWTIME[1] about it, titled "Advanced Hello World in Zig".
The TLDR is that Zig never had an easy builtin print function because it's important to let the programmer be precise about whether they want locking (or not) or buffering (or not). That's why the easy print function is in the debug namespace.
[1] https://zig.show
I'm a toe-dipper -- I need to run and play with some sample code (basic Hello, World stuff) to see if I like the ergonomics and wish to continue looking deeper, when an article is posted on HN and piques my interest to go try out (zig) the easy route is taken - install what Arch has as the latest and just test it out with the basics. ("The learn guide says I must have 4 spaces, what if I use 2? yeah OK that still works")
Take a look at other content by Andrew (https://andrewkelley.me) or from Zig SHOWTIME if you want to lazily evaluate Zig :)
Or whether the function accepts naked types as parameters or error unions!
The debug namespace is indeed the right place for a wholly generic print. The first day I took Zig out for a spin, std.debug.warn briefly confused me by eating errors and printing their name instead of (as I expected for a short minute) giving a runtime error.
This is a good design decision for the debug namespace, but you don't want a regular call to print() to take an unwrapped error without complaint!
https://www.drdobbs.com/learning-standard-c-as-a-new-languag...
The ziglearn website explicitly points to the website for this reason, although I agree it should probably include some warning about using the latest tagged release and how some of the tests will break.
The fact that debug.warn was renamed to debug.print during this release cycle makes this extra unfortunate, since it affects the hello world program.
Anyway, thanks for pointing this out, I will make sure to PR ziglearn to clarify this.
The result is that by default you get errors on windows with hello world programs. People will tell you to just change your defaults in your text editor to not write carriage returns. There seems to be a lot of rationalizations from a very unpragmatic choice.
This was a deliberate design decision by andrew for all zig pre-1.0 to supress discussions about coding style (use CR, LF, CRLF, LFCR as a line ending? tabs vs. spaces? ...) as it's considered bike shedding
Zig 1.0 won't have this restrictions anymore. More information can be found here:
https://github.com/ziglang/zig/wiki/FAQ#why-does-zig-force-m... https://github.com/ziglang/zig/issues/544
You're demanding 1.0+ usability from a language that's has yet to stand up on its own feet and is currently, unsurprisingly, 0.6.0 and won't be 1.0 any time soon because there's a massive amount of work that needs to be done.
This also isn't C and it wasn't developed/maintained for past 50 years.
None of the backwards rationalizations people give actually makes sense, especially since it literally breaks any "hello world" program on windows. It's insane that people would try so hard to make excuses for such a basic disregard for new people.
Think about the reverse. Let's say tabs are treated as spaces and carriage return bytes are ignored. Who would ever argue that something should be changed to break these things?
I admit that I don't know much about Zig and it does seem like a very interesting language, but at a glance it seems to me like the type system is significantly weaker than Rust's (which may be a good thing for compile times). For instance I see that they have special syntax for optional types (?) and for error handling (try/catch) when Rust can implement those using basic language constructs (Option<T> and Result<T, E>). Sure Rust also has the "?" sugar to make error bubbling nicer (like try in Zig) but it's just sugar, it doesn't do anything you can't do with normal Rust code (hence the older try! macro).
More generally I'm not really sold on their implementation of generics. It's the main reason I never gave Go a chance and Zig's version don't seem a whole lot better as far as I can tell. Rust's generics are complex and incur significant compilation overhead but they're also incredibly powerful.
Zig doesn't have generics in the strictest sense of the word.
What it provides is compile time execution and types as first class values at compile time.
You can write arbitrary compile time code that generates any type you wish, either with the traditional syntax or by using the @Type builtin (for example if you wish to generate a struct with fields that are defined by parsing some protobuf code, or w/e your heart desires, it is easily achievable).
For systems languages I don't think dynamic dispatch instead of monomorphization is not the right way to implement generics. And zig doesn't seem to be doing dynamic dispatch either. It shouldn't affect the amount of code produced.
The difference between rust and zig generics is difference between simplicity and rigour.
And I think having something built in is wrong, if it can be a library. Now library-instead-of-language-feature can lead to longer compile times and less friendly error messages. There are lot of tradeoff in language design.
Now that rust culture is formed largely of web / FP developers, it may indeed promote a template heavy, macro heavy programming style, which may lead to significantly longer compiles. For example, a struct based CLI flag parsing library instead of a simpler one, or map-filter chains instead of for loops. These may produce significant amount of generic expansions, and thus harm debug builds speed and compile times when overused. It also makes alternative implementations hard because the optimizer needs to be very good.
> Can you explain what you have in mind exactly?
Ok this is already long comment. Zig and Go leadership seems to be more focused on details of toolchain. For example this development, and zig cc. Zig also utilizes caching very well and focuses a lot on interop. That's impressive for a project at this scale. The focus on compilation speeds so much to create another backend is another example.
Rust is more about PLT and language/ tooling ergonomics. Cargo and popularity of VSCode among rust developers for example.
Not saying any of these are bad. Barring some overexcitement from rust community (I personally dislike that overexcitement), these new compiled languages are all very nice to have.
I'm split on this one. This sort of "axiomatic" philosophy of programming is fairly common, and it's not hard to see why; it just feels right, intuitively, for everything to be built out of a tiny set of simple constructs. It's really empowering to know that, if you understand those constructs, you can (in principle) understand any program. Lisp and Urbit are extreme examples of this.
But to everything, there is a price, and here the price is paid in performance, tooling, and readability. Reified builtins can offend our sensibilities, but they are easier to optimize, permit deeper tooling, and require less cognitive overhead.
Ultimately, I think it's a man/machine distinction. It's like...lined paper. Printers don't need paper to be lined in order to "write" in straight lines. A human, if they're being really careful, doesn't either -- but it sure helps. Lined paper is designed by meatbags, for meatbags. The question to ask is, who's doing the writing? I don't know offhand, but I'd guess that lined paper is far less common now than it was 50 years ago. Maybe in 50 years, it will be similarly uncommon to see programming languages designed around arbitrary human sensibilities. But even after that, I think most humans will continue to prefer arbitrary languages.
Not a rust programmer - but this seems odd. Map/filter/reduce are high level abstractions that describe most common operations you want to perform on sequence. It's non-trivial to infer from loop code (you need to scan at least multiple lines to confirm it's actually "just a map" or "just a filter", especially when you have more complex operation chains etc. and the mutable imperative nature of the code makes it even harder to reason about - more potential edge cases).
This is exactly the kind of stuff I don't want to do manually and expect the compiler to do at (near) zero cost - how can you "overuse" this ?
Re your points about toolchain: Rust is also developing a non-LLVM backend, and is one of the better compilers for incremental compilation
Probably that Zig is a systems programming language from/for people who understand and appreciate C - not C++/FP/etc...
To some degree, this is probably just because that's where the money is (A quick look at Andrew's blog shows lots of non-web stuff for personal projects prior to Zig), but the gravity of a day-job is fairly strong.
I'm a systems programmer by day, but my github profile indicates mostly web and common-lisp related things; I definitely look at my tools through that lens. The fact that I can get assembly-level profiling of my common lisp code is one of the things that keeps me on lisp vs. other dynamic languages.
For all the above reasons (and of course the obvious ones as well), it's a very positive sign for Zig that Andrew's day-job now is working on Zig. As a systems programmer, I do find it exciting, though I lack the time to spend on it currently (my first foray ran into a compiler bug after having written less than 100 LoC, which may have been coincidental, but certainly discouraged me from spending more time with it).
(Web dev is a totally cool and difficult engineering thing to do, it’s just disjoint from browser dev)
Anyway, Rust design is no longer influenced in any significant way by Servo, and most Rust contributors are not Mozillians
(So, everyone promptly hyperfocused on how to bootstrap the Rust Foundation in a hurry)
I'm a fan of Rust for things like a missile control system or a stock exchange, but not, say, writing a CRM. I'd gladly write a CRM in Zig however, given the proper library support and tooling. Why? productivity. In my experience it just takes 2-3x longer to get the same thing done in Rust vs. something like Zig/Crystal/Go and certainly vs. Ruby/Python/etc. For extremely mission critical code, this is fine, but for most startups I would argue it probably isn't appropriate.
Definitely staying tuned as Zig develops.
I was writing a emulator (and later: a compiler) in Zig which was originally written for PC. But i got the idea that hosting this stuff on the web page would be cool, so i learned how to do Wasm. The result was for both project a glue file with ~150 LOC Zig and ~150 JS and both compiler and emulator ran in the browser, and porting both projects took less than an afternoon of time.
Zig is a better C and C++. It should be used accordingly. It will do well in game servers, simulation servers and real-time systems that don't require hard safety constraints. Bonus points for Zig, it is a simple language and because of its nature it will be easy to build tooling around it to prove properties.
That's a bold claim. What's the C formal validator which automatically handles both memory management and concurrency in source of any size within seconds/minutes? (Without making the source its own dialect rather than C)
Anyway, you can verify the critical part of your C/Ada program by using: Why3 http://why3.lri.fr/ Frama-C http://frama-c.com/ (which is also extensible) SPARK https://www.adacore.com/about-spark
One company that I've worked for used generators, they were specifying the model with a DSL and they generated the code based on that. Can't give more information about that but there are others doing the same (SpaceX and NASA). Rust is very good for people that don't have the resources and money to throw at or are working in an industry where they don't need a certification, for example I have friends at Amazon in Romania that build Firecracker, a virtual machine that is supposed to improve the safety story on AWS Lambda.
And as an answer for your second question, at what cost? Also we don't know if Rust gets the scaling right. The compiler is already slow for the size of projects we already have.
Rust is brilliant, but for drivers, I would go for a language with OOM safety and strict static allocation, especially for embedded environments.
There a GC'd language with modern features should do best. Imagine OCaml but modern and great tooling. That would be easy for developers, more productive to write, less type errors, also more productive because of IDE support, and much faster than current crop of scripting languages.
Sadly, there is a vacuum for such a language. Go is too gruntwork and minor details, Rust is designed for high reliability system programming, Zig is designed as sytem language in spirit of C, OCaml is archaic and does not seem to improve by much.
The hopes in this area are
Nim - Nice python-like syntax, but feels a little unpolished, metaprogramming may be hindrance to IDE tooling and code readability on large codebases.
Crystal - Ruby like syntax and stdlib. LLVM compilation, very young language, no windows support, and (personal opinion) I think some work can be done on performance of idiomatic code.
OCaml - there is not much manpower behind it. All those Reason XYZ attempts to provide javascript syntax over it don't seem to have gotten traction. Tooling is pretty good considering how obscure language is. They might need a modern interface to toolchain, and an optimizing compiler seems to be being worked on. La k of multicore is often cited as a drawback but it is being worked on, and Python doesn't do multicore either, I don't think multicore matters to 90% of people doing webdev.
F# - Would be nice if it was not confined to .NET ecosystem and had good native compilation story.
I predict that there will be a paradigm shift away from GC in the next round of languages. I suspect what will win for something like web-dev will be a language with static memory management like Rust, but at a higher level, with slightly more tolerance for abstractions of non-zero cost.
There's still a market for more performant and predictable networked services. It's a bit too early for a full-fledged post explaining the feature in detail, but as far as I know Zig is the only language that can do async/await and at the same time ensure that when memory limits are reached, the system still behaves correctly (i.e. you can preallocate upfront the memory you need to instantiate a coroutine and, if that fails, you can gracefully return a 500 http error code).
Serious webdev is out there, and when SREs talk about the importance of predictability and low-variance in metrics, they aren't (just) writing content for marketing purposes.
Too much require.js look alike.
Also, how does this relate to the observation just above that "the right-most box is always invalidated ... the final executable is always re-linked from scratch ... since the final executable depends on everything else"? Does linking somehow get faster or easier with in-place patching?
And yes, it will be faster. The way Zig does this is by only recompiling and modifying the parts of the executable where the source has changed. Imagine a project like chromium, where your executable is 150 MB large. Now you change a single function. Classic linking would require the whole linking process and the write of 150 MB of code. Zig will only compile and patch a single function (when lucky: less than 1kB) and emit this. Less I/O, less overall work to do.
I see, thanks. I think I was misled by the article's "the final executable depends on everything else and so any meaningful change to the code will invalidate it". My understanding now is that, since the Zig compiler knows exactly what has changed, unlike a traditional linker it will not invalidate the parts that haven't changed. Is that correct?
If I'm not mistaken that refers to in traditional compiled languages.
I guess even worst case, writing 151MB to disk is fast if you have the data ready. Very fast with a modern SSD.
As noted in the article, the same strategy will be used for hot code swapping, which is one of the benefits of doing this.
This is indeed faster than generating new object files and relinking the whole executable. The video linked in the article shows a simple example with the current ELF incremental linker, where compiling the example program takes 1.5 ms while changing a string literal and updating the executable takes about 0.5 ms
Yes.
> Does it literally mean that the compiler opens the existing binary in a write-but-do-not truncate mode, leaves most bits in place, and only changes others in certain places?
I'm not an expert, but I believe the point is mostly about other computation required for "normal" linking, not just writing fewer bytes to disk.
> What other benefits is this supposed to have?
It makes the edit-compile-run loop faster when developing.
> Does linking somehow get faster or easier with in-place patching?
That's the idea. When code isn't position independent, moving a chunk of code requires recomputing a bunch of offsets, and similarly the linker needs to do more work when you don't have an indirect way of referring to functions etc.
Check out that 5 min demo video! I think it's pretty exciting.
I guess this is all in the pursuit of reducing link time, but a clean build will write the whole binary from scratch in a more compressed way?
Highlights:
- compile-time evaluation
- integrated testing through scoped blocks
- prevents some implicit casts that we take for granted in C
- catches allocation errors
- explicit error return handling
I appreciate it trying to make a better C.
That sounds excellent though, I'll check it out when it lands.
One example of this is the fact that codegen needs to be aware of the linker to emit correct code depending on the type of executable that is being generated, since different strategies can and are used for different formats (for example, currently the x86_64 backend will generate different instructions for function calls on ELF and PE/COFF vs MachO).
I would totally get behind LLVM or other compiler toolchains working towards this kind of functionality, but this is hardly something Zig developers could implement in a massive project such as LLVM (which is precisely why we are building our own :))
EDIT: To be clear, I mean the gap of slow build performance in the feedback look of code-build-check.
So, I welcome this news!
Don't get me wrong, I think he has a very fringe and suspicious outlook on LLVM and the US, but it is a big codebase that would be hard to audit for such things. Would Zig ever be fully independent and auditable by a small team when the break from LLVM is made? Just curious, and I would love input on the LLVM comspiracy topic.
[1] https://www.mail-archive.com/picolisp@software-lab.de/msg095...
LLVM monoculture is pretty bad though.. and its codebase quality is not very good either. But that's another issue.
I'm going to make the kind of prediction that will set me up to be the laughingstock of HN in a few years.
I think in-place binary patching will be the single most consequential development in build toolchains in the last twenty years; the most consequential development since a graduate student at the University of Illinois named Chris Lattner decided to embark on LLVM. If Zig is successful in this endeavor, by the end of this decade I expect that in-place binary patching will be a feature that comes standard with most build toolchains.
When you reason about build systems from first principles, it becomes abundantly clear that our current approach to linking is nuts. In both my current job and my last job, linking was by far the bottleneck in the development cycle, and a nearly unfixable one. Nothing puts a damper on getting into the zone like a sixty second link. Change a string literal somewhere? Wait sixty seconds. Fix a one-line bug in the test you just wrote? Wait sixty seconds. Change the guts of a function, but not its public interface? Wait sixty seconds.
Compilation is massively parallel, so if your compiler is slow, you have options. You can buy a bigger computer with more cores. You can outsource your compilation to the cloud. You can arrange your code cleverly to minimize dependencies, so changing one package doesn't result in invalidating too many downstream packages.
But if linking is slow, you're pretty stuck. Linking always happens, linking always happens last, and linking always depends on the output of all previous steps. Linking is rarely parallel and rarely incremental. You can switch from macOS to Linux, because Apple skimps on their linker. You can switch from the standard GNU linker (bfd) to the new GNU linker (gold) [0], which has both built-in parallelism and an incremental mode. But gold is only 20-30% faster.
I'm not excited by a 30% speedup. Saving 30% on a 60s link still means you have a brutal 42s link; still plenty long to kick you out of the flow. A 30% speedup is the kind of speedup that's erased in a year when your project is 30% larger. What I'm excited about is a speedup the size of several orders of magnitude! That's the magic that Zig is striving for here. Change a string literal? Bam! Your binary has already been relinked, before your finger even rolled off the return key.
This is the kind of thing that has the potential to change our relationship with compiled languages forever. The differences between a dynamic language like Python and a compiled language like Zig become mightly slim when recompiling a large Zig program becomes as fast as restarting a Python program.
And the only way to get there is to redesign the compiler toolchain from start to finish. You need everything from the design of the programming language down to a custom linker to cooperate. Binary formats like ELF are not designed to be hot-swappable like this; they're designed to be write-once. The same goes for debugging information like DWARF. They've developed some very clever tricks to make this work. I wouldn't be surprised if one day there's a custom, non-ELF binary format specifically for the use of incremental toolchains.
Huge props to Andy Kelley and the Zig team for having the vision and the guts to embark on this quest. (That reminds me: I am long overdue on setting up a sponsorship for the Zig Software Foundation [1] to fund this sort of work!)
Additionally, I'll note that gold's incremental linking is in-place binary patching, from what I can tell, and potentially another order-of-magnitude speedup even on your existing C projects - slide 11+ certainly appears to be describing such, and the caveats + resulting overhead: https://events.static.linuxfound.org/images/stories/pdf/lfcs...
(slide 17 also shows --incremental-base, if you decide you want non-in-place incremental linking)
And yeah, the `--incremental` support in gold is definitely very exciting. Unfortunately it is not complete enough that I've been able to use it on any of the projects where it would matter. Part of what gets me so excited about having first-class support in Zig is that it will work end-to-end, and adjust folks' expectations of how fast linking should be. Then those folks will go demand the same linking speeds of their C/C++/Rust/Go projects.
Building incremental support into LLVM/gcc/gold is definitely the long-term answer, but building an end-to-end incremental toolchain in Zig from scratch to prove the concept makes a ton of sense.
* 2x for linking rustc: https://blog.mozilla.org/nnethercote/2020/04/24/how-to-speed...
* 3x here: https://brokenco.de/2020/01/08/faster-rust-linking.html
* 2x faster than gold according to this: https://github.com/rust-lang/rust/issues/71515
Yes. My numbers are for a somewhat template heavy C++ project. I could see Rust/Go having different performance characteristics on that front.
http://neugierig.org/software/chromium/notes/2011/02/ninja.h...
One of the big tricks not mentioned there is to decompose the app into multiple .so files (only when debugging -- release is still a single binary), so that you don't need to rebuild files unaffected by your incremental changes. I wrote more about that here:
Composing a project of multiple shared libraries is a trick I've wanted to employ many times. Unfortunately neither Rust nor Go—the languages in which the two largest software projects I've worked on professionally are written—supports that model. (At least, not without throwing away the language-default build system, which has a lot of other costs.)
1. Even if you take a single binary and split it into multiple objects, you still must do the same "linking" work of resolving where functions point to. It's just that work is done as part of startup, at runtime, by the dynamic loader. I have seen (in other contexts) splitting up a big C++ binary into multiple pieces dramatically impact startup time.
2. You can't inline across shared objects. Inlining is the "mother of all optimizations" -- it's the biggest benefit, and most other optimizations interact with it. (For the simplest case, imagine a getter on an object. Inlined, it compiles to a pointer+offset dereference, while across modules it compiles to a lookup in a pointer table plus a function call.) This also means you can't delete unused functions and so on.
3. There's no real memory savings from shared objects in this manner, even when different subprocesses of Chrome only use different subsets of the binary, because the OS only pages in the part of the binary it uses.
In general, given there's no user benefit for multiple shared objects, there's little reason to not have a single binary, except for build time. But for releases Chrome does a lot of other work (I think even profile-guided optimization these days, though that is after my time) for performance anyway. In the old days we were making changes to save as little as 40ms of startup time (http://neugierig.org/software/chromium/notes/2009/01/startup... and http://neugierig.org/software/chromium/notes/2009/04/perf.ht...). Looking back at the post now, I see that -fPIC cost 12% of the WebKit JS engine performance, which is maybe another strike against relocatable objects (though maybe these days we use PIE anyway? I am pretty rusty on all this).
If I had an app that was multiple shared objects I might be more interested in measuring the benefit before switching it to a single binary, since while I know the benefit is nonzero you have to balance it against the cost of implementation. But in Chrome's case we started with the single binary build, so there was little reason to measure.
I would guess in IE's case they were more interested in providing reusable APIs used by multiple other systems. In Chrome's case that was never a design goal.
I wonder if a) this is still possible with Chromium today, and b) if/how it might be possible to incrementally build changed functions into new .so files, rewrite the symbol tables in the old .so files to garbage (if needed?), then relink (either incrementally/in-place or from-scratch) the executable. Dependency analysis and garbage collection could delete fully-superceded .so files.
I think (a) it's still possible, per https://chromium.googlesource.com/chromium/src/+/master/docs... .
(b) Creating new .so files that shadow functions in others is an interesting idea! I think you're implying that the executable wouldn't pick them up for free, which I think is true, but I'm not actually sure. I forget the details of how this works but it might be that you could just play tricks with the order of the files to make them shadow.
In practice with a fast linker (I used gold at the time, but I think lld is even faster than gold now) it doesn't matter so much as long as you keep the component libraries small enough. But my info is also nearly a decade out of date now...
On the other hand your way of thinking sounds a lot like what I understand Smalltalk image format to be like
That is exactly my model of the world, in fact. When linking incrementally, you don't want a tightly packed object file where the functions are stacked right on top of one another. You want something more akin to a key–value database, where you can swap out key–value pairs without corrupting the entire file. Whether that's an object file that "symlinks" a bunch of other object files, or is one large file with easily swappable blocks (like a SQLite database) probably involves evaluating tradeoffs that I don't have a lot of context on.
(For example, one benefit of keeping it all in one file is that you can send the binary to someone else or move it to another path without corrupting the file.)
That is kinda my point, we already have one perfectly good kv-database in the form of filesystem, so why not use that
We have a lot of container formats. Movie files incorporates video, audio streams, subtitles. MP3 files contains ID3, ID3v2 metadata. JPEG - Exif, PNG - metadata chunks, HTML - <script>, <style>, <meta>. ISO 9660 is a folder. ZIP is a folder with compression. DOCX is ZIP. These can be represented as folders and I would argue some users would like it a lot.
We use modules as files in python and ruby, yet it is a problem and docker solves it.
Zig specific - I do not think you advise to create library file for each function. As such it'd solve nothing - they'd have to patch libraries. And maybe they already have do that.
This. Possibly a whole paradigm shift in computer programming since the introduction of VM.
The linkers will do what they can to reuse all previously freed blocks of the file, but you can imagine a scenario where a function is updated but does not fit in the block that was previously allocated for it. In this case, a new block of the file will be allocated for this function.
If you then add a smaller function, it will occupy some portion of the old block. You can keep adding things into this block until some minimum amount of memory is reached (and as long as the declarations fit in there, obviously).
Repeating this process for all new declarations and for all the modifications of declarations, you can imagine some parts of the files will remain unused.
I hope this makes sense :)
But in general, the final executable will not be the same, this is correct.
Currently no work has been done on this as the focus is on the incremental linkers and codegen but I personally think we should be able to make a pretty solid tool that will not only provide langserv capabilities but also allow us to inspect and debug metaprograms with ease :)
If you remove those checks, while Zig does not give you a sound guarantee there will be no undefined behaviour in production, it makes testing easier and faster, helping correctness across the board. After all, the goal is not to write your program in a language with some guarantees, but to produce a program with as few bugs -- undefined behaviour or others -- as cheaply as possible. Which approach results in safer, more correct programs, Rust's or Zig's? Only empirical study will tell.
Reasoning is pretty simple: a dynamic approach (Zig) will only catch the memory errors for the tested code paths. This is probably only marginally better than running your C++ code's test suite with address sanitizer.
Rust's static approach provides memory safety for all code paths.
Since your point doesn't really have to do with which languages are involved (only that they are different somehow), one could similarly say that C might produce more correct programs than Rust or Zig for the same amount of effort.
New languages like Zig need to have a better story around safety than "only empirical observation will tell," because Rust's safety story is pretty compelling.
You're assuming much more. The costs I mentioned are not about being harder to code. C++ is also easy to code in.
> one could similarly say that C might produce more correct programs than Rust or Zig for the same amount of effort.
True, and right now it might very well be. C has exceptional verification tools, which is why a lot of safety-critical software is not written in either C++ or Rust, but in C, Ada, and some domain-specific languages that compile to C like SCADE. But assuming "all external tools being equal" I would guess both Rust and Zig would be better, because they both have a very strong focus on correctness and do something about it, while C doesn't even try.
> because Rust's safety story is pretty compelling.
Zig has an exceptionally compelling safety story as well. It focuses on safety no less than Rust, but it does so in a different way. My personal guess would be that Zig's approach to correctness is at least as good as Rust if not more so, though I could be wrong, and I have no problem with people guessing the other way. After spending years with formal verification and following software correctness research, my only conclusion is that we don't have anywhere near a good model for software correctness that would allow us to make any reasonable projection about approaches.
The story, though, isn't just about correctness, and languages can be compelling in other ways, too. Language preference is largely based on personal aesthetics, and aesthetically Zig and Rust are very different. Rust's aesthetics are not to my liking but Zig's are, just as I'm sure the opposite is true for others. I see no reason why there should only be one aesthetic approach in low-level programming just as there isn't one approach in high-level programming.
So not expecting any resolution on the correctness approach any time yet, I think those two languages would appeal to different people.
For not-so-reliability-critical pieces of software, eg game related, zig might be easier to get it done. Some people are really passionate about rust and very talented, they aren't affected by rust's non-zero cognitive overhead. But us trivial 1x programmers are.
Rust's approach is based on one guess re correctness, Zig's on another. My personal guess is that Zig's approach is ultimately more effective at reaching correctness, but I could be wrong. There's no way to know which is better by thinking about it -- correctness is just too complex a subject.
How is that not the goal? The more bugs I can statically guarantee are absent the better!
The goal is to produce a program with as few serious bugs as possible, as cheaply as possible. It is never a goal to use a programming language with certain properties; that could just be a means to that end.
> The more bugs I can statically guarantee are absent the better!
Only provided that those guarantees don't come at a cost that could make finding the bugs you don't statically eliminate harder. How could it make it harder? By making the language overall harder to understand -- both by informal and formal analysis -- making safe evolution more tricky, and by slowing down compilation, which slows down the testing cycle and reducing the number of tests you write.
Correctness is a very complex subject, and that "the more sound guarantees the better" is not a consensus opinion on how to get there. For example, even in formal verification circles, a lot of current research focuses on reducing soundness (e.g. concolic testing vs. model-checking/deductive proofs). It's like the question of how to catch more fish: is a smaller net with small holes that guarantees no fish can get through it better, or is it better to cast a wider net with larger holes, that might let some fish slip through but covers a wider area?
Perhaps in the case of the net and the fish an answer could be given ahead of time if you know everything there is to know about the distribution of the fish size and their dispersion, but that's not the case with software bugs. We simply don't know enough, and the only way to answer the question is through empirical observation, over a wide range of applications and over a long time. It's also possible that both approaches are equally good at achieving correctness, and then it's just a matter of personal preference based on other aspects of the language.
Zig is way easier to verify than rust code (no hidden control flow, no hidden allocations that might fail, ...) which means a security expert can read a single function (without knowing more about the code) and can reason about that function.
This does not work in languages like C++ and Rust where RAII is a common pattern and code will be implicitly be executed when some objects go out of scope.
Yes, sure. Rust does a lot of verification work already, but so do tools like cppcheck and other verification tools and i'm sure those will emerge for Zig as well.
Zig (in contrast to C or Rust) has no warnings. Behaviours that might not be OK are compile errors and cannot be ignored. There are also plans (as in "not implemented yet") to forbid declaring unused variables and such so even this gets a compile error. This makes Zig also easy to reason about (seeing a variable declaration means that it's guaranteed to be used later)
It's true that a macro in Rust could introduce an early, invisible return. But this can only happen inside a macro. A reasonable option would be to disallow those in a project where such concerns are critical.
Otherwise Zigs "try" is similar to `?` in Rust, and "defer" comes pretty close to non-obvious control flow for me.
I'm particularly not a big fan of "defer" (and "errdefer"), which I already don't like in Go. Sprinkle multiple of those in a larger function, and following the logic can become quite tedious.
That sounds very fraught with issues to me. Does that mean existing code will have to be updated in case other behavior turns out to be kinda iffy in practice? Isn't there a strong perverse incentive to just not add anything if that's the case?
What about the deprecation of existing code?
(Though, some people do use linters to get back the warnings. But the language itself doesn't have any.)
The main thing about having warnings isn't being silent in the face of possibly-shady constructs, it's about just making them errors, so you can be sure they don't occur. So e.g. in Go an unused import is an error.
Which is an incredibly annoying thing, because you can't easily just comment out some code and do a quick build to test something. You are forced to cleanup imports/unused variables every time you want to build something.
(Also, if you use something like goimports, it's pretty much automatic.)
{var | const } identifier : type
I know it's an extremely superficial observation, but it just doesn't feel good to me.
The style dates back to Pascal. I think it's better, personally, and that might be because Pascal was my first programming language, but I don't care, I'm just happy it's back in fashion.
As a side note, I was quite sure that it dated back to Algol 60, but looked it up for this comment, and nope! Type came before variable.
In a sense that it will feed your family and get you a job in the future.
Sure, Rust is even smarter and a savy Rust hacker is likely to run circles around the herds of mediocre Zig hackers.
But Zig hackers will come in legions while Rust hackers will die in beauty.
It's kinda crazy there's software that bothers the user to restrict CRLF. I tried zig and this was the first issue. I thought I needed to go find my AOL discs or something.
.\zig.exe build-exe .\hello.zig .\hello.zig:1:28: error: invalid carriage return, only '\n' line endings are supported
```
I just re-downloaded the 0.6 branch, and I get this error on latest Windows.
It's just one click in VS Code, but then I have to think about this every time I go between different files I might be working on.
It's NUTS to me that the compiler doesn't just regex them out or something, I can't justify using a technology that won't take even the smallest step for UX.
Imagine if you needed to set some character set or line-ending to bang out a python script, that'd be crazy, right? So, why should anybody bother doing it for Zig?
I want to like Zig, but I'm just not motivated to with an experience like this. I've been doing Python for a while, and I got my eyes on Nim for the next generation of languages.