Self-hosted x86 back end is now default in debug mode
ziglang.org
ziglang.org
The biggest pain point I personally have with Zig right now is the speed of `comptime` - The compiler has a lot of work to do here, and running a brainF** DSL at compile-time is pretty slow (speaking from experience - it was a really funny experiment). Will we have improvements to this section of the compiler any time soon?
Overall I'm really hyped for these new backends that Zig is introducing. Can't wait to make my own URCL (https://github.com/ModPunchtree/URCL) backend for Zig. ;)
There was a famous game with Lisp scripting, Abuse, and Naughty Dog used to have Game Oriented Assembly Lisp.
Maybe people should occasionally move away from their UNIX and vi ways.
Maybe when something better comes up, but since you never invested one single minute on improving Inferno we have to wait for another Hero ;)
MSVC++ is a nice compiler, sure, but it's not GCC or Clang. It's very easy to have a great feature set when you purposefully cut down your features to the bare minimum. It's like a high-end restaurant. The menu is concise and small and high quality, but what if I'm allergic to shellfish?
GCC and Clang have completely different goals, and they're much more ambitious. The upside of that is that they work on a lot of different platforms. The downside is that the quality of features may be lower, or some features may be missing.
Same with the FreeBSD Foundation (P: OS Improvements):
https://freebsdfoundation.org/wp-content/uploads/2024/03/Bud...
Other Foundations are more like the "Penguin Foundation".....
Not sure why but I was definitely getting some game of thrones vibes from your comment and I would love to see some competition but I don't know, Just code in whatever is productive to you while being systems programming language I guess.
But I don't know low level languages so please, take my words at 2 cents.
The fight for the Iron Throne, lots of self-proclaimed kings trying to take it... C is like King Joffrey, Rust is maybe Robb Stark?! And Zig... probably princess Daenerys with her dragons.
It's not perfect yet, but I can do C/C++/ObjC, Zig, Odin, C3, Nim, Rust, JS/TS, Python, etc... development and debugging all in the same IDE, and even within the same project.
Nevertheless, impressive that you can do so!
Excited to see what he can do with this. He seems like a really smart guy.
What's the package management look like? I tried to get an app with QuickJS + SDL3 working, but the mess of C++ pushed me to Rust where it all just works. Would be glad to try it out in Zig too.
SDL3 has both a native Zig wrapper: https://github.com/Gota7/zig-sdl3
And a more basic repackaging on the C library/API: https://github.com/castholm/SDL
For QuickJS, the only option is the C API: https://github.com/allyourcodebase/quickjs-ng
Zig makes it really easy to use C packages directly like this, though Zig's types are much more strict so you'll inevitably be doing a lot of casting when interacting with the API
But I did not know hashmap re-exported hashbrown, thanks.
looks like there's no way to access it, outside of hashmap.
Though maybe you just need the third party hasher and you can call with_hasher.
IDK man there's a lot going on with rust.
Even this is pretty usable, handling value conversions and such thanks to comptime. (Take a look at the tests here: https://github.com/eknkc/zquickjs/blob/master/src/root.zig)
Then everything went south, with the languages that took over mainstream computing.
$ # No modules
$ clang++ -std=c++23 -stdlib=libc++ a.cpp # 4.8s
$ # With modules
$ clang++ -std=c++23 -stdlib=libc++ --precompile -o std.pcm /path/to/libc++/v1/std.cppm # 4.6s but this is done once
$ clang++ -std=c++23 -stdlib=libc++ -fmodule-file=std=std.pcm b.cpp # 1.5s
Notice that this is an extreme example since we're importing the whole standard library and is actually discouraged [^1]. Instead you can get through the day with just these flags: `-stdlib=libc++ -fimplicit-modules -fimplicit-module-maps` and of course -std=c++20 or later, no extra files/commands required! but you are only restricted to doing import <vector>; and such, no import std.
[^1]: non-standard headers like `bits/stdc++.h` which does the same thing (#including the whole standard library) is what is actually discouraged because a. non-standard and b. compile-times, but I can see `import std` solving these two and being encouraged once it's widely available!
See regular discussions on C++ reddit, regarding state of modules support across the ecosystem.
Ok, the goalpost has moved on with what -O0 is expected to deliver in machine code quality, lets then have something like -ffast-compile, or interpreter/jit as alternative toolchain in the box.
Practical example from D land, compile D with dmd during development, use gdc or ldc for release.
Their algorithms were simpler.
Their output was simpler.
As their complexity grew, proportionately did program performance.
Not to mention adding language convenience features (generics, closures).
Generics were already present in CLU and ML, initially introduced in 1976.
Check their features.
real 0m18.444s user 0m17.408s sys 0m1.688s
On an ancient processor (it runs so fast I just never upgraded it):
cat /proc/cpuinfo processor : 0 vendor_id : AuthenticAMD cpu family : 15 model : 107 model name : AMD Athlon(tm) 64 X2 Dual Core Processor 4400+ stepping : 2 cpu MHz : 2299.674 cache size : 512 KB physical id : 0 siblings : 2 core id : 0 cpu cores : 2 apicid : 0 initial apicid : 0 fpu : yes
oh, and by the way that includes the package manager, so the compile time accounts for:
* HTTP
* TLS (including aegis-128l, aegis-256, aes128-gcm, aes256-gcm, chacha20poly1305)
* deflate, zstd, and xz
* git protocol
Do you have any metrics on which parts of the whole compiler, std, package manager, etc. take the longest to compile? How much does comptime slowness affect the total build time?
Well, one interesting number is what happens when you limit the compiler to this feature set:
* Compilation front-end (tokenizing/parsing, IR lowering, semantic analysis)
* Our own ("self-hosted") x86_64 code generator
* Our own ("self-hosted") ELF linker
...so, that's not including the LLVM backend and LLD linker integration, the package manager, `zig build`, etc. Building this subset of the compiler (on the branch which the 15 second figure is from) takes around 9 seconds. So, 6 seconds quicker.
This is essentially a somewhat-educated guess, so it could be EXTREMELY wrong, but of those 6s, I would imagine that around 1-2 are spent on all the other codegen backends and linkers (they aren't too complex and most of them are fairly incomplete), and probably a good 3s or so are from package management, since that pulls in HTTP, TLS, zip+tar, etc. TLS in particular does bring in some of our std.crypto code which sometimes sucks up more compile time than it really should. The remaining few seconds can be attributed to some "everything else" catch-all.
Amusingly, when I did some slightly more in-depth analysis of compiler performance some time ago, I discovered that most of the compiler's time -- at least during semantic analysis -- is spent analyzing different calls to formatted printing (since they're effectively "templated at compile time" in Zig, so the compiler needs to do a non-trivial amount of work for every different-looking call to something like `std.log.info`). That's not actually hugely unreasonable IMO, because formatted printing is a super common operation, but it's an example of an area we could improve on (both in the compiler itself, and in the standard library by simplifying and speeding up `std.fmt`). This is one example of a case where `comptime` execution is a big contributor to compile times.
However, aside from that one caveat of `std.fmt`, I would say that `comptime` slowness isn't a huge deal for many projects. Really, it depends how much they use `comptime`. You can definitely feel the limited speed of `comptime` execution if you use it heavily (e.g. try to parse a big file at `comptime`). However, most codebases are more restrained in their use of `comptime`; it's like a spice, a bit is lovely, but you don't want to overdo it! As with any kind of metaprogramming, overuse of `comptime` can lead to horribly unreadable code, and many major Zig projects have a pretty tasteful approach to using `comptime` in the right places. So for something like the Zig compiler, the speed of `comptime` execution honestly doesn't factor in that much (aside from that `std.fmt` caveat discussed above). `comptime` is very closely tied in to general semantic analysis (things like type checking) in Zig's design, so we can't really draw any kind of clear line, but on the PR I'm taking these measurements against, the threading actually means that even if semantic analysis (i.e. `comptime` execution plus more stuff) were instantaneous, we wouldn't see a ridiculous performance boost, since semantic analysis is now running in parallel to code generation and linking, and those three phases are faiiirly balanced right now in terms of speed.
In general (note that I am biased, since I'm a major contributor to the project!), I find that the Zig compiler is honestly a fair bit faster than people give it credit for. Like, it might sound pretty awful that (even after these improvements), building a "Hello World" takes (for me) around 0.3s -- but unlike C (where libc is precompiled and just needs to be linked, so the C compiler literally has to handle only the `main` you wrote), the Zig compiler is actually freshly building standard library code to handle, for instance, debug info parsing and stack unwinding in the case of a panic (code which is actually sorta complicated!). Right now, you're essentially getting a clean build of these core standard library components every time you build your Zig project (this will be improved upon in the future with incremental compilation). We're still planning to make some huge improvements to compilation speed across the board of course -- as Andrew says, we're really only just getting started with the x86_64 backend -- but I think we've already got something pretty decently fast.
I suppose it would compile faster if I didn't have symbolic debug info turned on.
Anyhow, our users often use dmd for development because of the high speed turnaround, and gdc/ldc for deployment with their more optimized code gen.
Some people say you should use an old computer for development to help you write faster code. I say you should use a new computer for development, and write the fastest code you possibly can by exploiting all the new CPU instructions and optimizing for newer caching characteristics.
Also, self-compile times are strongly related to how much code there is in the compiler, not just the compile speed.
I also confess to being a bit jaded on this. I've been generating code from 8086 processors to the latest. Which instructions and combinations are faster is always flip-flopping around from chip to chip. So I leave it to the gdc/ldc compilers for the top shelf speed, and just try to make the code gen bulletproof and do a solid job.
Working on the new AArch64 has been quite fun. I'll be doing a presentation on it later in the summer. My target machine is a Raspberry Pi, which is a great machine.
Having the two code generators side by side also significantly increased the build times, because it's a lot more code being compiled.
Never found a user who asked for that, either :-/
The MSVC limitations are maddening, from how short string literals must be, to the complete lack of inline assembly when targeting x86_64.
[1]: https://ziglang.org/documentation/0.14.1/std/#std.Target.Obj...
Anyhow, one of the curious features of D is its ability to translate C code to D code. Curious as it was never intentionally designed, it was discovered by one of our users.
D has the ability to create a .di file from a .d file, which is analogous to writing a .h file from a .c file. When D gained the ability to compile C files, you just ask it to create a .di file, and voila! the C code translated to D!
I somehow missed that D has that! I try to read the forums now and again, but I should keep more active tabs on how stuff is going :)
I don't know whether this is technically feasible, maybe you could run it on CPUs with good power management and force them to underclock or something.
Better spelled as Dlang!
When I tried compiling zig it would take ages because it would go through different stages (with the entirety of bootstraping from wasm)
Fast iteration time with incremental compilation and binary patching, good debugging should be the expectation for new languages, not something niche or "too hard to do"
The entire realtime rendering industry is essentially built on top of LLVM (or forks of LLVM), even Microsoft have switched their shader compiler to LLVM and is now (finally) starting to upstream their code.
The compiler infrastructure of most game consoles is Clang based (except Xbox which - so far - sticks to MSVC).
So all in all, LLVM has been a massive success, especially for bootstrapping new things.
Zig is pretty much exactly what I would want from low level language, I'm just waiting for it to be stable.
And, of course, kudos - I really appreciate minimalist design philosophy of Zig.
-OReleaseSmall -fno-strip produces a 580K executable, while -ODebug -fstrip produces a 1.4M executable.
zig's x86 backend makes for a significantly better debugging experience with this zig-aware lldb fork: https://github.com/ziglang/zig/wiki/LLDB-for-Zig
I don't recall whether it supports stepping through comptime logic at the moment; that was something we discussed recently.
I believe the most relevant links are https://github.com/ziglang/zig/issues/16270 and https://github.com/orgs/ziglang/projects/2/views/1?pane=issu... (as you can see, nothing is concrete yet, just vague mentions of optimization passes)
The compiler is fairly retargetable, this is an active area of work. So it’s maybe possible in the future to envision zig as an alternative compiler for fragments of the language.
Such as a more fine grained compile cache, better tooling to prevent i validations, removal of the world splitting optimisation, more use of multithreading in the compiler, automatic precompilation of concrete signatures, and generation of lazier code which hot-swaps in code when it is compiled.
- an integrated build system that doesn't use multiple separate arcane tools and languages
- slices with known length in Zig vs arrays in C (buffer overflows)
- an explicit optional type that you're forced to check, null pointers aren't* allowed (*when they are, for integrating with C code, the type makes that obviously clear)
- enums, tagged unions and enforced exhaustive checks on "switch" expressions
- error handling is explicit, functions return an error (an enum value) that the caller must handle in some way. In C the function might return some integer to indicate an error which you're allowed to completely ignore. What is missing is a standard way of returning some data with an error that's built into the language (the error-struct-passed-through-parameters pattern feels bolted on, there should be special syntax for it)
- "defer", "errdefer" blocks for cleanup after function returns or errors
- comptime code generation (in Zig) instead of macros, type reflection (@typeInfo and friends)
- caller typically makes decisions about how and where memory is allocated by passing an allocator to libraries
- easier (at least for a noob) to find memory leaks by just using GeneralPurposeAllocator
As someone who's always used higher level languages since I started programming and strongly disliked many of the arcane, counterintuitive things about C and surrounding ecosystem whenever I tried it, Zig finally got me into systems programming in a way that I find enjoyable.
While a quick compile cycle is beneficial for productivity, this is only the case if it also includes fast tests
Thus wouldn't it be easier to just interpret zig for debug? That would also solve the issue of having to repeat the work for each target
There is no real need to add an interpreter. Having custom backend s means that while currently it is being used for debug, far in future it might be able to compete with llvm for speed.
Adding an interpreter would be useless as u would still need to write a custom backend.
The problem is llvms slowness for debug and release.
The whole point of debug mode is debuggability, and hooking up such an interpreted Zig to a standard debugger like gbd or lldb probably isn't trivial, since those expect an executable with DWARF debug info.
PS: also acceptable debug-mode performance is actually very important, especially in areas like game development.
https://github.com/ziglang/zig/wiki/FAQ#what-is-the-status-o...
Nothing is decided for sure, but the plan is most likely to re-introduce stackless coroutines as a set of lower-level primitives [0], and support implementing the planned `std.Io` abstraction [1] using them. The usage isn't quite as "pretty" as our old async/await syntax, but this approach sidesteps a lot of design problems, allows applications to seamlessly switching between different async implementations (e.g. stackless vs stackful), and simplifies the eventual language specification.
It's true that we're not doing this work right now, but that doesn't mean async is "never coming back", nor that you'll be waiting "till 2028".
[0]: https://github.com/ziglang/zig/issues/23446 [1]: https://github.com/ziglang/zig/blob/async-await-demo/lib/std...
I’d say it’s really important to get async nailed before 1.0, because it’s potentially one of the biggest killer features for many projects.
Zigs async isn’t just about I/O, I’ve found the feature useful in hardware simulation code as well.
When choosing a new language (and ecosystem) in which to invest my time, I'm more likely to pick one that's versatile than one that struggles outside of a niche. Even with a background in processes, threads, manual event loops and callbacks, I find that higher level concurrency features make my life easier in enough different situations that first-class support for them is now fairly high on my shopping list.
Do I actually need a language with something resembling coroutines? No; I got by for decades with C and C++. But I want it. It makes me more productive and helps me to keep my code simpler, both of which are valuable to me. These days, I have found myself walking away from languages for being weak in this area.
I would like to contribute but faced difficulties because the compilation for all stage1/2/3 combined took a lot of time
* We want the final Zig binary it produces to be optimized using LLVM, and LLVM is incredibly slow.
* The start of the bootstrap chain involves a kinda-weird step where we translate a WASM binary to a gigantic C file which we then build; this takes a while and makes the first Zig compiler in the process ("stage1"/"zig1") particularly slow.
Luckily, you very rarely need to bootstrap!
Most of the time, you can simply download a recent Zig binary from ziglang.org. The only reason the bootstrap process exists is essentially so you can make that tarballs yourself (useful if you want to link against system LLVM, or optimize for your native CPU, or you are a distro package maintainer). You don't actually need to do it to develop the compiler; you just need to get a relatively recent build of Zig to use to build the compiler, and it's fine to grab that from ziglang.org (or a mirror).
Once you have that, it's as simple as `zig build -Dno-lib` in the Zig repository root. The `-Dno-lib` option just prevents the build script from copying the contents of the `lib/` directory into your installation prefix (zig-out by default); that's desirable to avoid when working on the compiler because it's a lot of files so can take a while to copy.
You can also add `-Ddev=x86_64-linux` to build a smaller subset of compiler functionality, speeding up the build more. For the other `-Ddev` options, look at the fields of `Env` in `src/dev.zig`.
And hey, I wrote a lot of the rendering code for that perf analyzer. Always fun to see your work show up on the internet.
FWIW there is a similar effort for Rust using cranelift: <https://github.com/rust-lang/rustc_codegen_cranelift>
Most of Zig's safety, or lack thereof, seems inherent to allowing manual memory management, and at least comparable to its "C replacement" peers (Odin, C3, etc).
Otherwise it is like getting Modula-2 from 1978, but with C like syntax, because curly bracket must be.
Zig is still not 1.0, theres not much stability guarantees, making something like Frama-C, even tho it is possible is simply going to be soo much pain due to constant breakages as compared to something like C.
But it is not impossible and there have been demos of refinement type checkers https://github.com/ityonemo/clr
Beyond that, tools like antithesis https://antithesis.com/ exist that can be used for checking bugs. [ I dont have any experience with it. ]
I also do not see how having decades of legacy software is holding anybody back doing new stuff in C in a better way. New C code can be very nice.
Microsoft had to come up with SAL during Windows XP SP2, Apple with -fbounds-safety alongside Safe C dialect for iBoot firmware, Oracle with ADI on Solaris/SPARC, Apple's ARM PAC extension, ARM and Microsoft's collaboration on Pluton and CHERI Morello, Apple, Microsoft, Google and Samsung's collaboration on ARM's MTE.
Lots of money being spent on R&D, for something WG14 considers exaggerated.
The new feature we will put in are not to enable safe programming, but to make it more convenient and to make safety demonstrable.
And I wish there was actually some real industry interest to pushing this forward. Industry seems more interested into claiming this an unfixable problem and we all have to switch to Rust, which gives them another decade of punting the problems with existing code.
Why doesn't WG14 prove the industry wrong then?
WG14 is a very small number of volunteers. It would help if the industry would actually invest development resources for safety on the compiler / language side and in cleaning up legacy code.
When all major OS vendors, some of whom are also compiler vendors, see more return into investment, contributing their money to alternative language foundations, or open source projects, than sending their employees to either WG14, or WG21, it is kind of clear ISO isn't going the way they would like to.
I would not call this an exaggeration, rather not listening.
Additionally, it would not surprise me if one of Zig, Odin, Rust eventually started popping up on console DevKits, or Khronos standards as well.
Several very popular, small, mature projects have zero or few open issues.
(And several mature, huge and unpopular ones too.)
Having the pointer nature of the operation locally explicit is way better than having to scroll back to find the type of the variable.
Unfortunely none of them came with free beer UNIX.
This design philosophy should lead to countless segfaults that are the result of Zig working as designed. It also relegates Zig to the small niche of projects in modern programming where performance and developer productivity are more important than resilience and correctness.
Sorry, what?
Code generation backends in the Zig compiler work by lowering an SSA-like structured IR called AIR to machine code (or actually first to another intermediate data structure called MIR, but don't worry about that). The thing is, AIR is intentionally quite high-level, the intention being that the code emitting AIR (which is complex and difficult to parallelize) doesn't have to waste loads of valuable single-threaded time turning e.g. a single line of Zig code into tens or hundreds of instructions.
However, this approach sort of just moves this work of "expanding" these high-level operations into low-level instructions, from the code producing AIR, into the codegen backend. That's good for compiler performance (and also actually for avoiding spaghetti in the compiler :P), but it makes it much more difficult to write backends, because you need to implement much more complex lowerings, and for a much greater number of operations.
To solve this problem, `Legalize` is a new system we've introduced to the compiler which effectively performs "rewrites" on AIR. The idea is that if a codegen backend doesn't want to support a certain high-level operation, it can set a flag which tells `Legalize` that before sending the AIR to codegen, it should rewrite all occurences of that instruction to a longer string of simpler instructions. This could hugely simplify the task of writing a backend; we don't have that many legalizations implemented right now, but they could, for instance, convert arithmetic on large integer types (e.g. u256) into multiple operations on a "native" integer size (e.g. u64), significantly decreasing the number of integer sizes which you need to handle in order to get a functional backend. The resulting implementation might not emit as efficient machine code as it otherwise should (generally speaking, manually implementing "expansions" like the one I just mentioned in the backend rather than in `Legalize` will lead to better code because the backend can sort of "plan ahead" better), but you can implement it with way less work. Then, if you want, you can gradually extend the list of operations which the backend actually supports directly lowering, and just turn off the corresponding legalizations; so everything works before and after, but you get better code (and possibly slightly faster compilation) from implementing the operation "directly".
If anything, it is a generation rediscovering what we have lost.
We been enjoying golang for the last decade and then some ;-)