Zig: software should be perfect [video]
youtube.com
youtube.com
This is more than a small piece of irony with a somewhat artificial example. Often, there simply isn't very much you can do with an error. In a SaaS world, you might not be able to tell the user what wrong, in order to not disclose some private information. You can log the error, but only if the network is available, and you can only log it to disk if the disk isn't full.
And then there are cases where you could try to explain the error to the user, but the reasons are very complicated and/or require intimate knowledge of the architecture to make sense, and/or require intimate subject knowledge to understand, or even obscure legal reasons.
Related: Only zig has fast errors with traces. Other languages have no traces or a high performance cost. Look up error return traces in the docs.
Out of curiosity, have you compared this to Rust’s backtrace mechanism? I’d be interested in the perf difference, if it’s available.
Looking at the source code https://docs.rs/crate/failure/0.1.3/source/src/backtrace/int..., "failure" uses https://docs.rs/backtrace/0.3.9/backtrace/struct.Backtrace.h... which is the callstack at a single point, not a trace of how an error was propagated.
Don’t let all the negative comments get you down. It happens to every language author. (See some of the nasty comments on Elm, Clojure, Go, Jai, etc.) I, for one, am happy that all of these languages exist, even those that don’t scratch my itches.
I like Rust and other modern languages too, but Zig strikes me as just about the best possible contender for a true C/C++ replacement (due to seamless interop, and a very practical and well designed simple language core — as opposed to the complexity of something like Rust, for example).
i.e. checked-exceptions?
I prefer a more decoupled/late-binding approach to error handling; so far Common Lisp does it the best I've seen.[0] The key insight CL people had was that often in the face of an error, you need help from somewhere away from your local context to figure out how to resolve it, but then for many cases, you want to return back to where the error has occurred and take the resolution path with your local context. In other languages that automatically unwind the stack to the exception handler, it's too late.
[0] http://www.nhplace.com/kent/Papers/Condition-Handling-2001.h...
Then it's "checked" but catching is optional.
F# is a language this is a lot like this, and I've recently been unable to get the editor with these features working in Linux, and it makes it rather horrible. If you had to remote in to a server and use vim, it could be rather horrible, and so on.
If we can get something like the language server idea working, really well, on all platforms and supported by all editors, then designing languages around certain IDE assumptions would more often be a good idea I guess.
It's a nice in-between something like a Smalltalk image-environment-IDE / Lisp Machine and a dumb IDE/editor that starts from source code and has to parse into AST into its own...
But that's entirely fine! Programmers are far too obsessed with exactly which functions trigger which errors when it absolutely doesn't matter. All you need to know is what errors you can handle and where in the code you can handle them.
If there is a network exception and can recover and retry it at the start of the operation, it literally doesn't matter which of the thousands of functions up the call stack could have possibly triggered it. The only thing you need to know is the top-level network exception and where your network processing code starts. And if you can't handle a network error, it literally doesn't matter if one was triggered. The best you can do is abort and record a really good stack trace for that type of error.
Surely also what errors can occur. Can the network using library throw disk IO errors? Permissions errors? Maybe it handles all the network errors internally to the library and I don't need to deal with those at all.
You don't really need to know in the specific what kinds of errors can occur. If it's possible to recover from an exceptional situation, it's only useful to know if that situation is possible so you can avoid writing code you don't need to. But there wouldn't be any harm in writing an exception handler for an exception that can't happen except for that wasted effort.
The flip side of this is I shouldn't need to worry about exceptions that cannot be thrown. When you say all you need to know is what you can handle and where, that list must be a subset of the possible list of things that can be thrown. There's no point worrying about whether I should be retrying something due to network faults if it never uses the network to begin with.
For your example if retrying network, I prefer to simply have a "ShouldRetry" property/interface on the exception itself since the triggering code has the best knowledge of how it should be handled. No need to know every possible network exception and sort them into retry or not retry.
> Is it something I can let the user retry if they know what's happening (e.g. IO error because the output folder doesn't exist)?
My favorite error handling is when you can just put a single handler at the event-loop of a UI based project. On exception you just show them the message and continue running. The stack unwinding code ensures the application maintains correct state and that the operation is unwound. If the user clicked "save" and a failure occurred they get the message and can retry if they want.
I admit there is an elegance to exceptions, but when I'm trying to write reliable code, I find reasoning about exceptions significantly increases my cognitive load. Error handling is a place where a little more verbosity is okay because it keeps my focus on the local context rather than having to consider the entire call stack. I basically agree with the TL;DR of the article: "My point is that exceptions are too hard and I'm not smart enough to handle them".
[1] https://blogs.msdn.microsoft.com/oldnewthing/20050114-00/?p=...
I used to think "this is hard, I need to learn about more systems until I'm smart enough to do this."
Now I think "this is hard, there's probably something wrong with the architecture. I'll fix it."
The idea that error states are perfectly knowable everywhere in the code is a myth. Even if that were possible at one moment, the instant anyone changes code anywhere it will immediately be wrong.
This is only true if your application has no state or invariants that could possibly be invalidated in the face of exceptions. For instance, what would be your solution to the 'NotifyIcon' example given in the linked article?
> The idea that error states are perfectly knowable everywhere in the code is a myth. Even if that were possible at one moment, the instant anyone changes code anywhere it will immediately be wrong
This applies equally to both error codes and exceptions. If a method N layers down in your call stack changes its behavior, that's a potential breaking change regardless of your choice of error handling.
The notify icon code is poorly structured to begin with. Simply creating a NotifyIcon object adds it to the UI? That's an awful design. If there was an add to UI step then it would be a non-issue; the half-constructed NotifyIcon object would never get added to the UI. This issue is not magically resolved by having to explicitly handle every error; you can make the same mistake with twice as much code.
> This applies equally to both error codes and exceptions. If a method N layers down in your call stack changes its behavior, that's a potential breaking change regardless of your choice of error handling.
I'm not talking about changing behavior, I'm talking about changing implementation. Behavior is part of the contract. But being able to safely change implementation is the fundamental principle of abstraction and is the basis for polyphorphism. If a method today does a calculation using a database but tomorrow is refactored to use a webservice -- as long as the contract/behavior is unchanged -- then the rest of the code shouldn't have to know about it.
Yes, and that's a very good thing.
> Many c++ projects disable exceptions entirely.
Yes, and they're objectively wrong.
> Writing "exception safe" code is tricky and non-obvious.
The word 'exception' here is redundant.
Exceptions make it much harder.
That's the whole point of exceptions.
That would indeed be ironic if that case was the only thing exception-based error handling entails. That is, if every program just let all exceptions go uncaught. Which is nowhere near why exception handling was invented, or how it's used in practice.
Does it still hold if you assume automatic resource (memory/locks/handles/etc.) freeing? (e.g RAII)
This is somewhat subjective of course, but from what I’ve seen, Zig has just the right set of features to modernize systems programming, without making the language too complex or difficult to write (which arguably Rust’s “borrow checker” system does), and (like Rust) gets rid of some huge legacy language design mistakes most people agree on today (e.g. nullable-by-default pointer types, or no way to know at compile time or at a glance what range of exceptions a function may throw).
And of course, the “automatic” interoperability with C is an essential part of any C/C++ replacement contender.
Rust, Zig, Kotlin, Swift, and many other modern languages can express the same concept of a null reference, but in a fundamentally superior way. In modern languages like these, the compiler will statically guarantee the impossibility of null dereference exceptions, without negatively impacting performance or code style!
But it goes beyond just static checking. It makes coding easier, too: You will never have to wonder whether a function returning a reference might return null on a common failure, vs throw an exception. You’ll never have to wonder if an object reference parameter is optional or not, because this will be explicit in the data type accepter/returned. You’ll never have to wonder if this variable of type T in fact contains a valid T value, or actually is just “null”, because the possible range of values will be encoded in the type system: If it could be null, you’ll know it and so will the compiler. Not only is this better for safety (the compiler won’t let you do the wrong thing), it’s self-documenting.
It blows my mind that any modern language design would willingly think nullable object references is still a good idea (or perhaps its out of ignorance), when there are truly zero-cost solutions to this — in both runtime performance and ease of writing code, as you can see for example from Zig or Kotlin.
[1] https://www.infoq.com/presentations/Null-References-The-Bill...
In any case I think the modern fashion of trying to eliminate null from PLs won't matter much in the effort to replace C, whereas something like a mandatory GC is an instant no-go (though Java at least was very successful at sparing the world a lot of C++). OTOH a language that makes more kinds of formal verification possible (beyond just type theory proofs) might one day replace C and have null-analysis as a subfeature too.
For example, most older languages with “the billion dollar mistake” have no complaint whatsoever when your write “object.method();” where it’s unknown at this scope whether “object” is null or not.
The fact that such code compiles is the billion dollar mistake; not the fact that the pointer is nullable.
I don’t care if you want to write nullable references everywhere, or whatever else you prefer or your application demands. That’s fine, so long as:
1. Non-nullable reference types must exist.
2. Nullable references types must exist as statically distinct from #1.
3. The compiler must not let you write code that assumes a nullable reference is not null, unless you check via a control flow statement first.
Now to take a step back, the principle behind this certainly applies beyond just nullability (if that was the point you were trying to make): Generally, dynamic, untyped invalidation states are dangerous/bad, while statically typed invalidation states are ideal. And yes, this does include bad states internal to a non-null reference, just as much as to a null reference.
Sum types are the key to being able to statically declare what range of values a function may return (or accept), and ensure at compile time that these different cases are all accounted for. If you aren’t aware of how elegantly sum types solve this, you should look into it — and I suspect it will be quickly clear why nullable references are useless, outdated, and harmful.
But at the very least, we’ve solved the pain of null dereference — and virtually without compromise. So, it’s irresponsible or ignorant IMO to create a new language that doesn’t include this solution in its core.
Well, you're in luck then! You don't even need a 'modern' successor, C++ (even the ancient versions) disallow null references.
For fairly low values of those. Creating a null reference is UB, your program is not legal at all.
The whole point is to NOT have it be accepted.
Which is a textbook example of the null reference problem.
Edit: There may be some terminological confusion here: when programming language folks talk about "references", they include in that definition what C/C++ call "pointers". See for example the Wikipedia article, which gives as the C++ example not C++ references, but C++ pointers.
Since nobody stopped you, the problem is still there.
That's useful, until you realise that all its smart pointers are semantically nullable (they can all be empty with the same result as a null raw pointer) and then nothing's actually fixed.
almost every language that gets rid of nulls with something like the Option type will let you still bypass it and get a null reference exception. Rust lets you unwrap, F# lets you bypass it. You could at least enforce a lint that doesn't allow the bypasses in projects where that is desired though.
It may seem to be just semantics, but it’s really quite important that the default (and most concise) way in these languages to read optional values is to check if they’re null/None first in an if statement, after which you can call “object.method()” all you like. It’s important that you can’t just forget this check; it’s essential to using the content of the optional, unless you explicitly type something like “.unwrap()” — in which case there’s almost no chance the programmer won’t know and think about the possibility a crash. Take this in contrast to the chance of crash literally every time you type “->” or “.” in C++, for example.
While I agree there are rare cases where .unwrap() is the right thing to do, I actually disagree here that it’s anywhere close to 10%: If you want to write a function that accepts only non-null values in Rust, you simply write it as such! In fact, this is the default, and no cognitive burden is necessary: non-nullable T is written simply as “T”. If you have an Option<T> and want to convert it into a T in Rust, you simply use “if let” or “match” control flow statements.
I actually think using .unwrap() in Rust anywhere but in test code or top-level error handling is almost always a mistake, with perhaps 0.001% of exceptions to this rule. I write code that never uses it, except those cases mentioned; while I’ve run into situations where I felt at first .unwrap() was appropriate, I took a step back to think of the bigger picture and so far always find safer solutions to yield a better overall design.
The cognitive burden from Rust comes not from this, but almost entirely from the borrow checker (a completely different toptic), and in some cases, arguably inferior “ergonomics” vs how Zig or Kotlin handle optionals.
For example, in some null-safe languages, you can write:
if (myObject) { myObject.mehod(); }
if let Some(x) = myObject { x.method(); }
IMO it’s not a huge deal, but certainly Rust could benefit from some syntax sugar here at least. Either way, both examples here are safe and statically checked by the compiler.
It's a tough balance. Rust could benefit from more sugaring, but on the other hand, Rust already has quite a lot of syntax at this point.
Yes, a Lisp-like language could do it. Why not Common Lisp? Because PreScheme and ZL were both better if we're aiming at C's niche:
https://en.wikipedia.org/wiki/Scheme_48
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.3.40...
http://www.schemeworkshop.org/2011/papers/Atkinson2011.pdf
I was looking for a way to do metaprogramming, verified programming, LISP-like programming, and low-level programming. Past few years of searching led me to a lot of verification stuff obviously, PreScheme for low-level Lisp, ZL for Scheme/C, and Red/System for a REBOL-like answer. I was eyeballing Nim, too, since the masses usually hate LISP's syntax. Then, I thought a front-end that let one go with various syntax's, esp inspired by famous languages, with the same underlying semantics or just easy integration.
On a practical note, I noticed that strong integration with the dominant language with little to no performance hit is extremely important. Clojure building on Java in enterprise space is an example. It reuses it's ecosystem. For system space, I started recommending using C's data types and calling conventions where possible in new language so calling it would cost nothing. Then, maybe an option to extract to C for its compilers. So, whatever above languages are created need to integrate with C really well.
It could, theoretically, be perfect. Since it's all theoretical at this point I mean.
At first glance, Zig vs Jai reminds me a lot of the Linux vs GNU Hurd thing (or a lot of other "worse is better" examples). A couple of geniuses locking themselves up telling the world "Just wait! It'll be awesome!" seldomly produces something that lasts.
of course, but sometimes it does.
My hypothesis is that Zig's compiler passes the equivalent of -march=native to the backend, which is why it should also be given to the C compiler to give a fair comparison (and a speedup of 30% or so).
It wouldn't surprise me if there are reasons Zig can do better or makes it easier to use better algorithms, etc, but you wouldn't expect that to show up in something as computationally straightforward as a cryptographic primitive.
By the way, as a C practitioner, I'm a fan of what you're doing with Zig; keep it up! C++, Go, and Rust don't appeal to me for all of the reasons mentioned in your talk.
I understand the place for rapid prototyping etc., and that not every software application deals with life-and-death situations — but even those that aren’t, I think our industry suffers a bit here. For example, to this day the Windows 10 start menu refuses to open sometimes (randomly) when I click on it, even multiple times. You could argue that this isn’t a huge deal, because within 30 seconds it usually “fixes itself” (or something like that), but it still doesn’t shake the overall feeling that we’re tolerating way too much shoddy software in 2018 than we should.
Or in terms of performance: I know not every application needs bare-to-the-metal speed, but something feels wrong with the world when my “supercomputer” (compared to a 1990s PC, for example) literally lags when I’m typing or scrolling in some apps, when a 1990s era PC could respond to essentially the same content interaction with almost zero latency.
Some few decades ago, we had far more klunkier programming languages, far slower hardware, and yet somehow yielded better tangible/functional results in some cases. Therefore, I’m very much in favor of anything that moves us towards higher quality software, and Zig (and Rust, and others) are all exciting examples of that.
It's clear what his point is.
One of the two examples he cited in his introduction, the Android one, wasn't even caused by an unhandled OOM error. Android by design will kill unused processes if they're occupying memory that the foreground process needs.
If we want software without bugs, removing hidden allocations in languages is far down in the priority list.
Most languages make allocations behind your back so that your code can focus on the logic you actually care about since you could not do anything about running ot of memory anyways.
However there are projects where that level of control matters. For those projects C is currently still the default choice, even though it was designed more than 40 years ago. Some choices made back then might be huge liabilities for code we are writing now, because we still need that kind of language. A modern alternative to C could provide a huge value to all of us, mostly through more correct, secure and/or performant software.
What about other run-time exceptions, like divide by zero? Are they checked?
What about Hoare's billion-dollar mistake (null pointer exceptions)? Does Zig have non-nullable references?
Pointers can not be null. However you can have optional pointers which are guaranteed to use the 0x0 value as null and have the same size as normal pointers.
Of course many projects don’t use them, for various reasons.
I wonder: is turning on full compiler warnings, then fixing them - is it making my software better, or just satisfying some type of neuroticism?
One warning I recently disabled on $workproject is -Wtrigraphs.
No compiler I care about has had them enabled by default in more than two decades.
> Documentation is about 80% done
And yet there is not standard library documentation. :(
Such a language could be visualized as strict-by-default Haskell (with type-classes, uniform pattern-matching, minimalist syntax - everything, except monads) plus ideas from Erlang, Go and Ocaml.
Perfection and imperativeness, it seems, does not match.
I'd expect some downvotes are based on negative reactions to this part of the comment: "the title is as dumb as dumbness itself." (Dear avip: if your comment said "the title is off-base" you would have made your point just as effectively and without the downvotes.)
* https://en.wikipedia.org/wiki/Undefined_behavior
* https://en.wikipedia.org/wiki/Timing_attack
* https://en.wikipedia.org/wiki/Thread_safety
* https://en.wikipedia.org/wiki/Privilege_escalation
Some of these probably don't come to everyone's minds right away. Please share your favorites.
The behavior(s) that a particular language guarantees is a design question. Once those guarantees are specified, we can objectively evaluate a particular language in terms of how well it does according to its own standards.
However, I interpret the message from the video as "Let's make it really easy to achieve perfection".
In other words if we keep improving development tools and technologies, we may eventually be able to achieve perfection in each individual project nearly for free.
Whether this is realistic or not, I do not know.
I prefer to say it this way: I want software to adhere to a contract. That implies that we want people that use the software to understand that contract. To be more precise, I'd say that:
(1) a good contract defines the scope of correct behavior.
(2) a contract may (or may not) give some bounds (or constraints) about what happens outside of the scope of correct behavior
Edit: the original title does not mention perfect.
The "original title" you're referring to is what the person who uploaded the talk to youtube titled the talk, not what the speaker titled the talk.
And you’re reaching a bit far. You can write a perfect function that adds two 32 bit integers. There is a subset of code that can be written perfectly. Especially if you don’t need Turing completeness to write it.
Zig just tries it’s best to make it easy to write as much as possible of low level code in a perfect way.
how ? the problem of adding two 32 bits integers is itself imperfect since you may at some point have big integers to sum, so any solution is inherently flawed, too
Given two 32-bit signed integers, produce a 32-bit signed integer that is a sum of the inputs; if the sum doesn't fit into 32 bits, wrap modulo 32 bits.
Given two 32-bit signed integers, produce a 64-bit signed integer that is a sum of the inputs.
Given two 32-bit signed integers, produce either a 32-bit signed integer that is a sum of the inputs if it fits into 32 bits, or an error code indicating that it did not fit.
The problem with languages like C is that they let you be imprecise and get away with in. If you just run with "add two signed 32-bit integers", and implement it as "x + y" in C, it will compile, and it will run, but the result is undefined behavior if it overflows (note: it doesn't even mean that you get the wrong value - it can literally do anything at all). In Zig, if you write it as "x + y", it will panic on overflow, which is at least well-defined and fails fast; and the language provides you with tools to implement any of the three properly defined scenarios above in straightforward ways (i.e. you can add with wraparound, or you can add with an overflow check).
Smalltalk solves that problem by promoting the result to arbitrary precision arithmetic. It does the same for integer division. For example, 5 / 2 returns a Fraction, not an Integer.
There's a large number of JS bugs and sandbox escapes in all browsers. They certainly can crash the user device.