Zig ELF Linker Improvements Devlog
ziglang.org
ziglang.org
I am still a little salty about `@cImport` removal, though! without it, I can't confidently call it "Kotlin of C" anymore.
dvui?
That sounds good on paper. But as a guy who tried to learn Kotlin and only it. It comes with baggage to learn Java to use its libraries because... You know... they interact seamlessly and stuff. In the end, for a new learner, it might actually make things harder.
Nothing about Zig and C here, just a bit salty from my experience with Kotlin.
This is the same place F# has been stuck in. It’s a great language on its own, but you can’t just use F#. Every F# must also do C# interop. It’s too 100% optional in theory, but never in practice. The best CLR/JVM libraries for anything are Java/C# ones. You need to interop with them to develop practical Kotlin/F# applications. You can limit yourself to the Kotlin/F# ones, but then you’re artificially limiting yourself to experimental libraries at best. You will find yourself needing a charting library, a DNS library, an SMTP library, an AWS SDK or a rabbitmq SDK. The best ones are gonna be Java/C#. Yes, you can always find a random GitHub repo for a “Kotlin-native X”, but the Java X library is a thousand times more mature, stable, performant, feature rich, etc. Same problem with F#. And the “glue” code is so “straight forward”, why would any one bother?
In contrast, your statement about F# strikes me as mostly true - albeit my interop was always the other way around (consuming F# code from C#).
There is a reason Android team has Java friendly guidelines.
And JavaScript comes with the baggage of learning about web browsers and NodeJS's "fs" module.
That is why I always say keep with the platform, and why despite my endless rants on C, I keep myself up-to-date in regards to it.
Eventually they always diverge.
From all guest languages on the JVM, the Kotlin community is the one I rather avoid, as many behave thankless to the platform that actually makes their toy possible in first place.
Clojure, Groovy and even Scala, focus primarily on being nice with Java and having a symbiotic relationship.
Koltin folks are like "we're replacing Java, ahaha stupid language".
Well where is the Kotlin Virtual Machine, other than Android userspace, which nonetheless requires Java support to benefit from Maven Central ecosystem?
Certainly driven by Android folks, I would say.
this makes porting projects gradually, file by file, rather cumbersome. now I have to rewrite quite a lot of Chocolate Doom because my port was halfway there and then @cImport got gutted... or keep going with Zig 0.16.2 until it's either 100% Zig or has little enough files that upgrading won't make my build.zig file implode in lines of code
Didn't Go already do that?
> I am so used to thinking that Zig, Rust, and the likes are only viable in niches where C is viable, but no. not anymore at least - once this linker and incremental compilation on other targets land, Zig will become THE C replacement
Yes, and it will still only be useful in the same niche that C is because the entire philosophy of Zig is to essentially be like C. You're never going to interate at JS/Python speeds with Zig because you'll always be wrangling with memory lifecycles, object lifecycles, etc...
Rust is significantly different.
Zig is for when you need control over the allocator (also many such cases).
no. GC pauses turn any serious systems work into hell.
> Yes, and it will still only be useful [...]
this does not exclude the possibility of creation of libraries that manage everything for me within their domain of responsibility, such as dvui
did you miss the context I was responding to? They were comparing against JS & Python and rapid iteration speeds. That's obviously not "serious systems work" where GC pauses matter
> this does not exclude the possibility of creation of libraries that manage everything for me within their domain of responsibility, such as dvui
Sure, but if you're philosophically okay with hidden allocations and control flow, why not just choose a language that doesn't compromise itself for those ideals in the first place?
that was me comparing, please kindly read usernames next time :)
> why not just choose a language that doesn't compromise itself for those ideals in the first place?
I might want to wrap a timing-sensitive machinery into some nice UI. Rust has Tauri for that, sure, but now we are bringing npm and have zero chances of having GPU-accelerated UI without a crap-ton of fuckery which would be easier in Zig. another path for resolving that same issue is Compose Desktop + Project Panama, but then you are dealing with data marshalling, FFI boundary, and manual resource management in an environment that does not expect this.
so, here is a genius idea: why not have everything in one language? C++ already does that, much like C, but Zig does that so much better, and incremental compilation time is one of the more practical and immediate developer UX benefits it provides. Rust? good luck having shared mutable state there.
Pretty much correct, yes.
> Zig will become THE C replacement and that will let me iterate at the speed of JS or Python with performance of C or Rust.
Fundamentally impossible. C/Zig/Rust have 100% performance as a top goal, which has to be traded off with something else and that's always realistically going to be programmer work/effort/time.
You can't have a house built 100% fast but also 100% cheap and with 100% quality. At best you could cleverly abuse the law of diminishing returns and aim for ~80% in all three areas. That's basically what Go's trying to do.
> once this linker and incremental compilation on other targets land,
In any case, why would a better linker and faster compile times of all things achieve this supposed goal?
Beyond being low level, Zig is still pretty memory unsafe and has you make choices about each allocation, making it unappealing as an applications language. Zig and Python are completely different worlds.
That’s essentially what technological development does, is make those tradeoffs more favorable.
And technological improvements don't remove these tradeoffs. You still have to balance tradeoffs. The best thing you could do is have a mix of technologies that choose different tradeoffs. A single technology (zig) won't do.
then how does Zig achieve ~90%?
> In any case, why would a better linker and faster compile times of all things achieve this supposed goal?
sub-100ms rebuild is actually more important as the project grows. when you iterate, you think differently. picking different colors or fonts or whatnot becomes much cheaper, so you are more willing to try
Since you ask, the front end is self hosting in NQP and with the ripening RakuAST project increasingly in Raku Grammars. The new AST (6.e.PREVIEW) will bring much better introspection and high level optimization handles. So the potential to refactor/rewrite the VM for substantial speed gains is wide open.
Anyway those with skills and interest are welcome to join the -Ofun at https://raku.org/community
Zig focus on compilation speed and give developers control (even more so than C). Raku, as a Perl descendant is a giant toybox and there is no one telling you not to use them. Both have in common that they give a lot of freedom to developers, which is really enjoyable. They also have in common that they are not very mature and have a limited market share and fine with it, and a community that looks genuinely nice.
The opposite of Rust. The language has the "bondage and discipline" philosophy, kind of like ADA, the idea being that it does everything it can to stop you from making mistakes. There is a lot of value in that, but it is not particularly fun. Its community was similarly defined by rules, its code of conduct was infamous, again, it serves a purpose but to me, putting the rules forward doesn't make the community look like a fun place to be. And there is the evangelism, the Rust community is aggressive "rewrite it in Rust!", "no memory unsafe languages!", etc... I have never seen such attitude from the Zig community. Sure they love their language and will tell you it is the best in the world, but they will not say that you are wrong for not using it. As for Raku, they don't seem to care that no one else use their language, they just hope it will happen eventually if they continue going forward.
This is more of a meme than reality at this point.
Good luck and happy hacking!
Is any of that already lying around in Zig?
The Zig team between 0.16 and this has really made me glad I chose Zig as the target instead of Rust - which probably would've been a lot easier to target (since it's already memory safe).
I believed it had the best build system design and was the best transpilation target, and I really believe that 6 months later.
The main reason I wanted no GC is because I think aliasing is the root of all evil, and I want a language with zero global complexity (but doesn't require a PhD to use).
It's not clear how much concurrency is part of what you're trying to solve.
All I could find is this: https://blorp-lang.org/docs/concurrency/ - which doesn't give me much as to how you handle shared memory, safety, deadlocks, etc.
Definitely down to chat more - looks like you've got some traction, which is impressive and awesome!
I'd love to pick your brain as it appears you're further along than I am.
What are the over-arching goals of your language?
1) I want to minimize global complexity, which by definition maximizes local reasoning.
2) I want to make the vast majority of bugs simply unrepresentable - taking it past Rust, and even past Pony - WHILE allowing shared mutable memory, but without requiring a PhD to use.
The goal is in EASY mode, it's barely harder to use than Ruby or Python (just the occasional pedant compiler error that has automatic options to fix itself most of the time). You don't even have to supply types or compile. It has a REPL, etc.
When you bump to DEFAULT mode and then to STRICT mode, all the annotation is automatic - your code just might look "ugly" if you like having no types anywhere etc.
But DEFAULT & STRICT mode give people and LLMs everything they need to know to understand the effects of an individual function.
WRT to inference, yes. I infer everything in EASY mode.
And the compiler give the user autofix via choice when a type is ambiguous (I don't default to huge union types - I assume no one wants to do that and make them choose a type - may potentially allow AutoUnion to allow that).
I couldn't tell if you're using affine ownership, but I assume so if you don't have a GC. If they try to create an alias - they get a use after move error, and the compiler tells them they need to either COPY (auto-fix) or create a RefCount (usually auto-fix) - they pick.
How do you prevent data races in concurrent code?
Or do you not allow shared mutable memory?
If you want some feedback, I'd recommend putting some concurrent benchmarks on your home page (and if you have some already - I'd clearly separate them).
When I originally scanned it, I just assumed this was another predominately sequential language with no good concurrency story.
If you're actually competitive with Rust/Tokio/Crossbeam and Go on the most common concurrent patterns, then you've got a really compelling project!
I suspect if you don't cherry-pick benchmarks, you're going to run into some performance problems with not allowing shared mutable memory - though maybe you can avoid most of that if you have some type of built-in actor pattern.
But if you're actually competitive across the board with Go & Rust/Tokio/Crossbeam - I'd love to take a deeper look, because that is NOT easy to accomplish.
I didn't see any of that from a cursory look at the language, though.
Comptime is very powerful and awesome to use for a transpilation target.
Also, Io and how Zig handles allocation and gives you full control of the allocator make it super easy to do things you probably want to do in your language.
But I assume that any kind of incremental linking, is mutually exclusive with link-time optimization? I.e. you'd never want to use this option for a release build?
It has been done before.
And LTO is when the C people and the C++ people started to agree.
Also obviously it is about how fast the actual implementation of the compiler/build-system is.
Definitely true when using VC++ with C++20 modules and MSBuild.
It also helps not compiling everything from source as many UNIX folks do.
Lol, sorry, but as soon as MSBuild is involved the compiler can be infinitely fast and you'd still need to be waiting for the build. Also the main problem of MSVC is the slow linker, and that isn't fixed by C++ modules. This is also the first time I'm hearing that C++ modules actually help with compilation speed in real world projects - the best I've heard so far is that they're a bit faster than precompiled header but not by much, which simply isn't good enough for typical C++ projects.
The company behind Cadifra UML Editor is quite happy with their migration, the owner keeps posting about their modules experience on Reddit C++.
Microsoft also has CppCon talks on the matter.
All my C++ hobby projects use modules, as I only care about VC++.
You keep posting about obscure products like whatever UML Editor that noone cares about, why would I listen to what they say, and why do you not even link anything to look up?
Well, somewhere down in one of those blog posts,
https://devblogs.microsoft.com/cppblog/integrating-c-header-...
You can maybe even do it in C if you are generating code using scripts or abusing macros.
On the other hand, you can write sane C++ (without stdlib) and clang or gcc is able to compile it extremely fast. Because clang and gcc are really fast.
You can write a rust program without dependencies and it would be extremely hard to make it compile slow unless you are abousing generics or procedural macros. But you can write a 100 line webserver using some framework like axum and it will compile really slow. This is not because Rust is slow to compile or rust compiler being slow.
On the other hand you can write a zig file with a single test and compiling it using Release mode takes 3+ seconds on a modern computer because Zig compiler is extremely slow when using LLVM backend. This is also obviosly not because Zig is a slow language to compile since their own backend (non LLVM) has super fast compilation.
Iteration speed is everything now, if and only if you learn from the additional iterations.
32743 lines; 0.953s
Will the Windows side for 0.17.x get some compiler improvements as well or is this Linux only?
thank you, this helps!
The strict definition being we don't count developments that happened before version 1.
Like when we talk about Rust, we don't mention the virtual threads or GC or the @ symbol for GC references. Even though those all happened during its development.
also, nitpick: they said zig has been around for ten years. this is, strictly, correct. the zig project has existed for ten years, just like how rust has existed for about 20, now. a project still exists if it is pre-1.0. nobody was talking about versions before you.
The author is strongly biased and opinionated on his architectural and design choices, and those choices resonate with many.
I would faster get back to Modula-2 or Object Pascal.
It's evaded other linkers in the past: gcc, llvm, mold etc....
It's the opposite: people have become more receptive to communication about this work now that there's "drama" attached to it.
This post I co-authored with Andrew is from 2020. In it we announce the idea of getting rid of LLVM from the debug build pipeline and since then work has been steadily going forward, it's just not trivial to bootstrap a full compiler pipeline for all major targets, but we're finally getting there.
What if that’s true and what if that’s not true?
which applies to contributing to the Zig project.
The "contributor poker" blog post should probably be a required reading to understand where it comes from: https://kristoff.it/blog/contributor-poker-and-ai/
"Anti-AI stance" is painting it with too broad of a brush. You're definitely not breaching any CoC or whatever by using AI for your Zig projects.
yes, that was my point. as domeone who uses ai extensively to write zig (and someone who has made very small non-AI cobtributions to zig in the past), rejecting ai is currently a strategically good decision for core zig.
In any case, I'm super glad for this milestone (and impressed!).