Zig 0.8.0 Release Notes
ziglang.org
ziglang.org
I recently arrived at a similar design for manipulating NFAs as adjacency matrices, as a replacement for more pointer-y adjacency lists, by way of sparse matrix data structures. Rather than Zig's performance angle, I found that it made whole-graph operations much easier to implement and reuse.
I've also seen similar approaches from array languages like APL, for example this project for running this kind of stuff on GPUs: https://github.com/Co-dfns/Co-dfns
And, I've seen it in Rust as a "workaround" for the borrow checker, where that framing tends to make it feel like cheating or settling, which IMO is unfortunate since when people arrive at it for other reasons it seems to have a lot of other benefits!
There are probably more contexts I'm not familiar with- anyone have any good examples from domains they've worked in?
Co-Dfns is Aaron Hsu's project, and he has a talk[1] on a compiler implemented in a nano-pass style, once in Racket, once in Chez Scheme[2] and once in Dyalog APL, each using CPU and GPU.
For compiling an AST with ~16 million nodes, he gives benchmark figures that Dyalog APL takes 64MB RAM to hold it, Racket takes 1051MB and Chez Scheme takes 1485MB. His provocative statement for the talk is "pointers are the refined sugar of programming". The APL version is 1/50th of the lines of code[4], runs typically 9x faster on the CPU, 50x-200x faster on the GPU, despite being interpreted by Dyalog vs compiled Scheme. There are benchmarks where the APL version loses in runtime, under 1k lines of code (which he calls "tiny tiny") so lots of startup overhead and not enough big arrays to gain ground on.
He has one talk (available on YouTube) about his approach to using arrays to working with tree structures in a high-performance way in APL[3].
[1] https://www.youtube.com/watch?v=UDqx1afGtQc
[2] he used to be on the Scheme R6RS steering committee, so he's not new/inexperienced with it http://www.r6rs.org/steering-committee/election/electorate.h...
[3] https://www.youtube.com/watch?v=hzPd3umu78g
[4] 17 lines of code which took a PhD researcher (him) 5-10 years to write, mind you.
The other one is that, separating the thing from the representation of the links between things is not unlike a set of basis functions forming a high dimensional space. The indices into the store forming a point in that space.
Some say Petgraph is a hack, I think Petgraph is a mathematical sculpture.
Great point! I've been using Rust for 4–5 years now and putting objects in Vecs and using integers as "pointers" felt a lot like cheating (i.e., "I'm doing the wrong™ thing!"), but every time I tried to "fix" things to use references my code ended up uglier, harder to understand, and sometimes slower.
In time, I've learned that it's perfectly fine way to structure data. And as you point out, it makes it easier to work on a large structure structure itself rather than having to go through its individual components (e.g., you can manipulate all the nodes in a graph with a simple for loop rather than having to do a DFS/BFS traversal).
Some of my favorite highlights from this release:
> With the advent of the Self-Hosted Linker, Zig is now able to generate well-formed and codesigned binaries on arm64 macOS (aka the Apple Silicon). It is also able to cross-compile C, C++, and Zig code to an arm64 and x86_64 macOS. Additionally, arm64 nightly binaries of Zig are automatically generated by our Continuous Integration service, meaning both arm64 and x86_64 macOS are now Tier 1 targets.
From what I know, with existing cross-compilation for C/C++/Rust, code-signed MacOS binaries are not possible. This new Zig linker makes it possible and works for C/C++/Rust as well[1].
> Finally, as a side experiment, Jakub added in Zig Build System integration with Darling (#8760), a translation layer of macOS syscalls to Linux, with the intention of being able to cross test MachO binaries and macOS specific tests directly on Linux simply by passing in an additional flag -Denable-darling to zig build test.
This line of thinking - not only "how do I cross-compile", but "how do I test my cross-compiled code?" is super interesting to me. I hope it is explored further.
Finally, it's just incredibly cool to see how quickly Zig is developing and how financially stable it is[2], supporting handfuls of core team members to work on Zig. I haven't seen any other language community achieve this, let alone without corporate backing. Truly beautiful.
[0] https://devlog.hexops.com/2021/increasing-my-contribution-to...
[1] https://www.reddit.com/r/rust/comments/nii64t/zig_makes_rust...
It speaks volumes that zig can provide toolchain-level benefits to other low level languages like Rust.
As far as I know, this is possible with clang and the new mach-o lld port as well.
While it's possible with clang/lld in C++, it looks like it is much easier with zig.
https://github.com/indygreg/PyOxidizer/tree/main/tugger-appl...
Whether you should teach Rust to code-sign your software using this code is probably a tricky question, but it appears to me that you could.
EDIT: Aaand shockingly, I'm not the first to come up with this idea: https://github.com/ziglang/zig/issues/782
https://twitter.com/zigshowtime/status/1399634302833471491?s...
I don't follow LLVM development. Is this normal for the project?
It's definitely a real problem[0][1][2][3], I would argue LLVM needs a longer release-candidate cycle before a release is cut - to give downstream users more to pick up and report regressions, as well as LLVM contributors to fix them. A higher quality bar would be great.
[0] https://twitter.com/andy_kelley/status/1365186310088974336
[1] https://twitter.com/andy_kelley/status/1303046926430908416
[2] https://www.reddit.com/r/rust/comments/gh3thc/llvm_10_has_pe...
[3] https://www.reddit.com/r/rust/comments/n5ne5i/regression_mis...
From Rust's point of view infinite loops are well formed. Sure, your program never exits but that's what you told it to do, so it's your problem not Rust's.
However in C++ that isn't a thing. In C++ an infinite loop is Undefined Behaviour. And so LLVM just didn't have working infinite loop code. If your C++ program has an infinite loop, too bad the ISO Standard says your program is wrong, clang is behaving as intended. Sure that's crazy but C++ programmers have a sort of Stockholm Syndrome.
So LLVM had a bug, but because the most important customer of LLVM is clang, and clang didn't care about this hideous bug while its users had just become numbed to the consequences, the bug lived on for ages.
So hopefully just temporary transition pain while the new manager learns the ropes.
Quality really went down with that RH guy.. :/
Well, this says that zig team reported those bugs as "release blockers".
Doesn't mean the LLVM release team also considered them release blockers (or that it had too).
Besides, whatever the "process" says, isn't it ultimate up to the release manager?
I've been sponsoring for a couple months now. I love the passion here and want to support a smart, dedicated guy leading a project in the open. I'm curious just to see what happens and where the project goes. Watching Andy brainstorm, experiment, and triage in GitHub issues has been worth the $5/mo, in project management tips alone!
One of these days I'll have to actually give Zig a try...
> git actually has a simple design, with stable and reasonably well-documented data structures. In fact, I'm a huge proponent of designing your code around the data, rather than the other way around, and I think it's one of the reasons git has been fairly successful […] I will, in fact, claim that the difference between a bad programmer and a good one is whether he considers his code or his data structures more important.
In fact, we had it in mind when we designed TigerBeetle [1][2], a financial accounting database to do a million financial transactions a second.
We started with the data structures, made sure that these were one or two cache lines to reduce cache misses and perfectly aligned to eliminate false sharing, and then we added built-in accounting primitives to move the code to the data, instead of the data back and forth across the network to the code, which is how these kinds of balance tracking systems that we were working with before were typically implemented.
The result is much simpler, a perfect replicated state machine, and also much safer, because the business logic and data are executed within the state machine, protected by the distributed consensus protocol (Viewstamped Replication Revisited by Liskov and Cowling), rather than a mess of SQL transactions spanning multiple distributed systems, all with different clocks.
[1] https://github.com/coilhq/tigerbeetle
[2] Guess what language we wrote it in? And this is making more and more sense, like a secret weapon. The community is amazing, deeply talented, with great taste, and at the vanguard of where things are going for these kinds of systems.
Having never worked with a language that is particular about memory, Zig has felt like a nice balance. Dealing with the allocator is clear and being intentional about the bit-width of things like integers helps articulate what is expected, which can be important in an emulation project. The built-in WASM compilation seems like something I will explore with this project too. That part is a nice touch
> Additionally, it is doubtful that the new backend will allow for seamless cross-compilation to macOS since every macOS binary is required to be a PIE and link dynamically against libSystem dylib, which will require the lib's presence on the host for the lld to reference and link against.
This is true, but it's a platform requirement, not a linker requirement. How does zig's linker get around this? Does it produce static binaries (LC_UNIXTHREAD instead of LC_MAIN)?
As far as cross-linking is concerned, we do the same thing we do for macOS libc headers - we ship the definitions. You can think of it as effectively shipping a preprocessed version of libSystem.tbd with Zig. This is still early days though, so currently as a workaround for not having a functional yaml parser (so that `zig ld` can link against tbds), every unresolved proxy symbol is simply assumed to come from libSystem. This is of course not ideal since then unresolved symbols are flagged only at runtime rather than at link time.
Anyhow, shipping preprocessed libSystem.tbd makes it possible to cross-compile valid PIE binaries (which is a strict requirement on Apple Silicon anyway).
If you wanna discuss it more, feel free to DM me in Zig's Discord - I'm always game to discuss linkers!
It took me about two weeks to feel comfortable & productive in it.
Are there strong products and parsers, services, actor models, interfacing with Middleware written in Zig? Or is the theme - drop to C based interfaces to those products? What's the zen of Zig say?
What are some notable wide use applications, solutions written in Zig?
The idea of including a C++ and a C compiler gratis is brilliant, and solves a big interoperability headache very elegantly.
For my own purposes, IIUIC it is fatally flawed, lacking destructors, but as a raw C replacement it has legs. Compile-time execution and types as regular compile-time values makes it actually powerful.
seriously? write a `deinit(self: *Self)` function into your struct and explicitly call `defer obj.deinit()`. Explicit is better than implicit. And the penalty? One line of code per destroy site, plus writing the `allocator.destroy(self)` line in your deinit method. As a result of being explicit, SO many headaches are solved. What are the rules again when you have a C++ class that polymorphically inherits destruction? What were the override rules, if your child class needs to implement its destructor? What order do destructors trigger when there's more than one for stack-objects? what about in std::vector?? What about in std::map??
I'd love to see it if it can be implemented in a way that fits the language, but that remains to be seen.
Destructors are necessary for correct-by-construction coding. The only code you can be completely certain is correct is the code you didn't have to write. None of dnautics's questions ever arise. I don't have to think about them, and I can't get them wrong.
If you imagine destructors have something to do with memory management, you have much yet to learn.
Zig is doing almost everything right to become essential infrastructure, but will need to add some equivalent of destructors to get there.
Not true at all, there are always compiler errors, and in many cases you can statically check your code using an external validation system. I'd be 99.999% confident of most of the WRITTEN code of SEL-4, probably more confident than ANY code "not written" in rust, because it's been formally verified by a theorem-proving external system that enforces far more invariants that the rust compiler. To be dramatic, the rust compiler after all will not throw a fit if you write
fn add(a: i32, b: i32) -> i32 { a - b; }
Correctness also can be other things that you might not be able to lean on the compiler to check so easily. For example, cryptographic side channels (timing e.g.). If there is ANY ambiguity caused by the inexplicitness of the code you produce, maybe because you're not thinking about the cost of the destruction event, because, after all, you didn't write it, and there's not lexical artifact left over, it can get extremely challenging to audit for those sorts of things.
When your case is so weak, it is better not to present it.
To be fair, if we come back to this opening argument, I'm still wondering how strong the premise here is? Are destructors in fact essential to essential infrastructure as you claim in your view? I'm not sure that I'm convinced by the case as you've presented it so far. I don't think we need to look too far to find counter-examples of essential infrastructure with the highest levels of safety that achieve this safety without destructors, and where destructors would be precluded by the safety model.
Also, as someone who's done a fair amount of security work, the timing safe argument against hidden control flow is at least compelling and something to explore and not ignore out of hand. If a language design can make crypto simpler and safer to implement correctly, then I would say that's already a good indication of what the language may be able to do for less complex domains.
It is very limiting to think that destructors is the only way to solve resource scoping problem.
Both defer (and more generally scope guards) and context managers certainly do, to say nothing of try/finally.
Outside HN, CppCon and C++Now bubbles not all C++ codebases are as clean as we would wish them to be.
If you want to use RAII for heap allocations you obviously need some sort of RAII wrapper for them. And smart pointers are the usual solution. But technically you can do without if you really really don't want to make dereference overridable, you can just have the "smart" bit (RAII / automatic deterministic deallocation) without the "pointer" bit.
I have seen plenty of new std::string("some") and other similar crimes in offshored code, or code without ownership that gets updated by whatever external contractor got brought in to implement 5 Jira tickets that finally got some budget assigned to them.
To be fair this isn't exclusive of C++, I have seen similar bad practices in other languages, C++ just gives more opportunities for them.
Right, but we're not discussing C++ are we? We're discussing RAII. C++'s mistakes are C++'s mistakes, adding RAII to a language doesn't imply you have to replicate C++'s mistakes, nor are C++'s mistakes issues with the concept of RAII, or destructors.
From type theory point of view, ideally RAII aware types should only compose with outher RAII types without escape hatches.
Rust and Ada do it better in this regard.
Yes, new is itself too low-level, so std::make_unique("pjmlp"s) is the modern idiom; and std::string is limited (expect unicode-aware strings in '23). But a heap-allocated string object is a thing equally easy and useful to express in Rust. (Dunno about Ada: does it have any analog of destructors, yet?)
You have to remember to `free` the string explicitely.
> Yes, new is itself too low-level, so std::make_unique("pjmlp"s) is the modern idiom
Is it? Isn't that going to create an `unique_ptr<char*>`?
> You have to remember to `free` the string explicitly
That is not a type problem. It is far from clear what pjmlp means here. unique_ptr should have been invented in 1985? Time machines have their own problems.
For arena allocated memory, you have to manage the lifetimes manually just as you would for C -- except that the arena itself can be lexically scoped, at least.
C++ is not a perfect language by any means, but destructors are not a mistake IMO. I too find having to manually "defer" cleanup annoying. I'm still interested in zig for the comptime features, though.
While that is true, I did not consider it worth mentioning given the context of the discussion. I still don't.
> Also destructors tie design to stack frames.
No more so than linear types.
This reminds me of Douglas Adams.
"The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at and repair."
This seems to be a matter of taste with no absolute conclusion to be reached which style is better. At least until we get significantly better at analysing codebases and software quality.
... until you have to write a class yourself. Then you have to think of the complexities introduced by destructors such as copy semantics, move semantics and the destructors themselves.
You also have to deal with architectural questions such as should my file have a close method (C++) or do I always leave this to the destructor (Rust).
If I was to give an opinionated summary, a little complexity at the call site has been moved to a lot of complexity in the class implementation. Since there is, on average, one class implementation to many object uses, destructors might be considered a win. But I don't feel so strongly about it that I wouldn't consider a language for not having destructors.
What makes a language powerful is the ability to put essential semantics into a library where they can be got right once, so all users of the library benefit. It is worth design, implementation, testing, optimization effort if the results can be delivered over and over, in many different contexts, reliably.
Destructors are the necessary ingredient to pull transaction semantics into libraries. Without, you are left to write comments explaining users' responsibilities, and users have to read and act on them perfectly, or be wrong. With, you can code them correctly once and users have no opportunity or temptation to get them wrong.
Any interesting code involves custom classes written by less experienced developers that are only used once or twice in an application -- it is there that that the complexity of destructors hurt.
None of those complexities spill over onto destructors. They were perfect in their original conception, and are unchanged 40 years on. They evoke no regrets. No bugs trace to anything unfortunate about destructors.
That is a specific language issue. Rust doesn't share most of these concerns for instance, the only complexity is writing the Drop itself (which I agree can be subtle), but a type can't even be both Copy and Drop (and Rust has none of the Rule of X complexity of C++).
> You also have to deal with architectural questions such as should my file have a close method (C++) or do I always leave this to the destructor (Rust).
That seems less like an architectural question and more like a stylistic question.
> If I was to give an opinionated summary, a little complexity at the call site has been moved to a lot of complexity in the class implementation.
Except that's not actually true, much of the complexity you'll find in a destructor (excluding the complexities inherent and exclusive to C++ which nobody else is under any requirement to reproduce) will have to be implemented as whatever "cleanup" method is deferred. And then you need the defer system, and you need to actually call it which you can forget.
So while destructors are definitely not trivial to add to the language, defer spreads the responsibilities and complexity arounds, and increases the occurrences of possible errors, as well as the number of failure modes.
It also adds semantics ambiguities / complexities to the language: when you defer an expression, what is evaluated when? Dtors don't have that issue, you construct something according to normal language rules, then the dtor will be called at destruction.
C++ did it first, having been born at same place as UNIX and C, all major C compiler vendors quickly added it into the box.
Generics, lambdas, concepts, templates, exceptions, ....
- https://ziglang.org/learn/overview/#integration-with-c-libra...
- https://ziglang.org/documentation/master/#Import-from-C-Head...
https://bootstrappable.org/ https://bootstrapping.miraheze.org/