Building an efficient and portable programming language with Zig
fastly.com
fastly.com
I hope it stays this way. And maybe it can given the way the foundation has been bootstrapped, and that the project is driven largely by a single creator. Having the funding be driven largely by small donations of actual developers is a healthy incentive structure.
Zig, next to Rust, is one of the more interesting language projects going on these days. Since Rust became orphaned from Mozilla, I am a bit ambivalent about the amount of investment in the project which is being driven by tech giants currently.
And to be honest, I don't care that much about who is behind a tech, as long as they are making the right choices. (except if it is Apple)
Programming languages getting a bunch of investment from its users is open source working possibly at its best.
It's only recently after it became very popular all tech giants started investing in it.
Similar are many other languages like Ruby, Haskell etc.
Rust is developed within a corporate world of Mozilla, like Go language in google, so its natural it will embrace corporate world and companies like Microsoft, Amazon which will in time decide it's future directions.
Guido also worked always for quite well known companies after he left academia.
But I like to see the emergence of a few sane proposals to replace C/C++, I've been extremely impressed so far by the work done by the team behind Zig.
The best feature, from my perspective, is that Zig does not intend to be a better C++ but a better C.
i have nothing against having strong opinions, i am the biggest offender but completely disallowing some features, some of the best features static typing can offer is just nonsense.
every language designer should study common-lisp and start with the operators : and :: these two operators alone tell you that the language designed for practical use.
for me overloading the operator + and function 'add' is exactly same. you should then forbid function overloading as well now. you can argue about the weakness of operator overloading but disabling completely, not even a workaround?
why not just enable oo and have safe alternatives instead? . safe(a + b) . a s+ b
When trying to understand a codebase, it is much easier to not worry about hidden indirections.
being able to misuse a PL i believe is not a bug but a feature. we really suck at predicting the future.
But it is better to avoid giving them too many ways to shoot themselves in the foot.
Forcing the code to be explicit is good design, in my opinion, simply because it is much easier to write code than to read it, we have to put more weight on the clarity side.
Meta-programming can be super powerful and practical, and the current trend of reusing the language itself is clearly better than previous attempt.
But meta-programming in itself is not a novel idea, C macros and C++ templates are past well studied examples, and both produced awful results.
I am not sure they can prevent a new cycle of this nightmare.
I have no doubt about how powerful metaprogramming is, but it makes me feel that understanding and contributing to libraries that use it is out of my reach.
I have found that when any project gets to a certain size, it's almost inevitable that metaprogramming will be required, unless you want to make everything super dynamic and sacrifice performance. The idea of being able to do metaprogramming in the language I used to write the program itself is an interesting one.
What's wrong with templates ? They give you a compile-time functional language that operates on types, isn't that great ?
With something like that:
var x = a [+] b;
There is no possible mistake.
It is probably better to have native support for small vector operations, as this can also help to streamline SIMD optimization.
In practice (as a video game dev) I only use operators overloading to have infix notation on vector maths.
"It's really easy to make fun of C++, but everyone has that one feature of C++ that they like. And they want to put it in Zig. If everyone had their favorite feature from C++ in Zig, Zig would just be C++. But I'm not going to let that happen."
a language with first class cross-compilation alone is enough for me to respect you as a language designer and zig have so many features like this. i am a huge fan.
i have only one favourite feature in c++ and it is templates/ctfe. without the function/operator-overloading templates are incomplete, i would then use c instead. i am still using c++ with all its failures, with all its complexity, with all its uglyness|ineleganceonly because c++ got one thing right. you can at least somehow modify the language.
edit: i have to point out that i have never worked in a big team (more than 10) and you can (probably should) safely ignore my ideas/opinions.
In a marketing interview with biased questions, anything is possible.
TBH, I have a positive impression of zig, but this kind of interviewing doesn't help it.
You could roll your own 'safe' memory pool in C such that use-after-free doesn't produce undefined behaviour. You'd still have a bug, but it wouldn't have to invoke UB at the level of the C programming language.
https://www.joelonsoftware.com/2004/06/13/how-microsoft-lost...
> I first heard about this from one of the developers of the hit game SimCity, who told me that there was a critical bug in his application:
> it used memory right after freeing it, a major no-no that happened to work OK on DOS but would not work under Windows where memory that is freed is likely to be snatched up by another running application right away.
> The testers on the Windows team were going through various popular applications, testing them to make sure they worked OK, but SimCity kept crashing.
> They reported this to the Windows developers, who disassembled SimCity, stepped through it in a debugger, found the bug, and added special code that checked if SimCity was running, and if it did, ran the memory allocator in a special mode in which you could still use memory after freeing it.
It's not supposed to be. In most cases, memory allocation is an operating system concept. As a systems programming language, zig must not abstract this away from the programmer.
Side note: the proposal at [0] seems very close to [1].
[0]: https://github.com/ziglang/zig/issues/782
[1]: https://www.haskellforall.com/2013/06/the-resource-applicati...
Even if Zig doesn't bring safety by default to the language, however, I think you could get it in practice by making the standard library enforce safety. Then, unless someone goes out of their way to write their own libraries from scratch and eschew all of the safety mechanisms, they would likely live in this safety bubble.
> even Rust programs don't give you sound guarantees for memory safety as many of them depend on unsafe code that isn't soundly proven
There's no such thing as 100% memory safety; at the extreme end there are bit flips caused by cosmic rays. But the evidence suggests that practical language-enforced memory safety is actually worthwhile, despite the fact that no theoretical absolute memory safety is possible. All memory-safe languages have runtimes and FFIs, which in Rust is the unsafe blocks. This doesn't change the fact that, empirically, memory safety, even the imperfect memory safety we have to live with in the real world, is a meaningful improvement in stability and security.
> That the cheapest way to write such programs is to have sound guarantees for everything is an interesting hypothesis, which would get you a language like, say, Idris, but it is not the consensus.
Straw man. Nobody is talking about proving all code correct. What is reasonably describable as consensus nowadays is that having the language enforce memory safety is worthwhile. The idea that enforced memory safety has, in your words, "more downsides than upsides", is increasingly at odds with the consensus.
To get concrete, I see no reason to believe that quarantine (Zig's current solution to UAF) is a meaningful solution to use-after-free problems, given that quarantine has been deployed for a long time in production allocators in other languages and has failed to eliminate this bug class in the wild.
I am saying that it may come at a cost, and overall it may not be worthwhile; but, of course, there are different kinds of safety bugs, different kinds of soundness, different costs, and different alternatives.
> But the evidence suggests that practical language-enforced memory safety is actually worthwhile, despite the fact that no theoretical absolute memory safety is possible
What is it that the evidence suggests exactly? There are so many variables. For example, Zig's runtime checking could be much more effective than similar systems for C or C++, because it is much easier to track all pointers in Zig, just as its overflow protection is far more effective than sanitisers in C, because there's no pointer arithmetic (unless using unsafe operations) and all buffer sizes are always known. So you can't compare what Zig can do to what C can do. Then there's the question of what the safety mechanism is. Then, for each point in this coordinate space, what is the fitness you're looking at? Reducing a particular bug or improved overall correctness?
> What is reasonably describable as consensus nowadays is that having the language enforce memory safety is worthwhile.
No, this is not true, or at least, that depends on what is the fitness function and what exactly it is compared against. To put it bluntly, there is absolutely no consensus that Rust would more cheaply produce programs that are more correct overall than Zig. It's possible this is the case, as is the opposite or that the two are about the same, but we just don't know yet.
> To get concrete, I see no reason to believe that quarantine (Zig's current solution to UAF) is a meaningful solution to use-after-free problems, given that quarantine has been deployed for a long time in production allocators in other languages and has failed to eliminate this bug class in the wild.
But that's not how we look at correctness. The goal isn't to eliminate all UAF bugs. Of course completely (or close to it) eliminating a specific bug is more effective at reducing it than not completely eliminating it. If my only goal was to eliminate UAF, then I'd choose Rust's approach over Zig's. But usually our goal is to write a program that meets our correctness level for all bugs combined in the cheapest way possible. Investing a huge language complexity budget in completely eliminating UAF, as opposed to, say, reducing it by 99%, has not been shown to be the best way to achieve what we actually want.
I am not saying that it's not reasonable to believe that soundly eliminating this particular bug with something like Rust's ownership and lifetime system ends up being better overall, but it is equally reasonable, based on what we know, to believe that Zig's way is more effective overall, or that the two end up the same. There's just so much we don't know about correctness.
See "Memory safe iBoot implementation" section,
https://manuals.info.apple.com/MANUALS/1000/MA1902/en_US/app...
For Zig I really don't get what it offers over Modula-2 other than a more C like syntax and compile time metaprogramming.
Maybe I will be proven wrong.
(note: "we" excludes me personally but I mean it as "us who work and study comptuing at large")
"Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interests of efficiency on production runs. Unanimously, they urged us not to--they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous. I note with fear and horror that even in 1980, language designers and users have not learned this lesson. In any respectable branch of engineering, failure to observe such elementary precautions would have long been against the law."
C.A.R Hoare on his Turing award speech in 1981.
Authors just chose to ignore what was already out there and do their own thing instead.
UNIX's success in the universities made the rest.
the documentation is still scattered. The official website [1] is a good starting point. and I wish I'd seen this [2] earlier (I hope this gets incoporated into the official resources)
[1] https://ziglang.org/learn/
[2] https://www.lagerdata.com/articles/an-intro-to-zigs-integer-...
This is probably the most comprehensive resource: https://ziglang.org/documentation/master/
You can find documentation on the language website, also there is an active community and you can ask questions direction on Discord.
- That an out-of-memory condition is elegantly handled or not.
- That an out-of-memory condition is properly reported to the caller.
- That is simplifies WebAssembly support; whatver you put in the allocator your could put into malloc.
- That it easily allows arena allocators; you can only use an arena allocator if you intimately know every single uses of the passed in allocator, allo the implementation details of the libraries you call and of *all* its dependencies. If a dependency, for example, implements a cache, the arena allocator will corrupt it when deallocated.Zig’s clearly about a more homogeneous approach where I am happy to have multiple pieces of syntax to cover different usecases.