Have we learned nothing? Many languages without NIL/Nil/nil/null/NULL existed already when Golang was born.
And this attempt to heal the damage hurts, too:
> “make the zero value useful” philosophy
This is like instead of programming by exception (Java), it's programming by ignorance (of errors).
I like a few things about Golang (handling of numeric literals, for example), but not this thing.
In some respects, you can get around this by simply adding another field. You can have `bool over18; bool over18IsSet` and I'm guessing that Google's internal usage of protobufs does this.
In a certain way, even getting rid of null/default values doesn't fix all problems when it comes to things like updating data. Think about updating a record where a field could be set or unset - let's say a person's age could be a number or empty. If I want to send a request updating their name, maybe I send `{"name": "Brian"}` because I don't want to update the other fields. How do I unset Brian's age? `{"age": null}` makes some sense, but a Java (and many other language) deserializer will have null for age with `{"name": "Brian"}` too. I mean, the age field has to be set to something in the Java object. You could manually read the JSON, but that's janky and brittle - and hard in terms of interoperability with libraries and languages.
Maybe Google's protobuf designers would argue, "you really need to have explicitness around your values and forcing defaults means forcing engineers to deal with things explicitly."
I don't think I agree with that. I don't like Go's nulls and default values. I think most languages are moving away from that kind of behavior with other new languages like Kotlin and Rust going against it and older languages like C#, Java, and Dart trying to bolt on some null-safety (Java via the `Optional` object and C# and Dart via the opt-in `TypeName?` similar to Kotlin). It's possible that this is a wrong direction chosen by many languages. We've seen bad programming language fads before. In this case, I think we're on the right track and Go's on the less-great side.
Go has a lot to like, but this is one of those odd decisions. I understand why they did it. Go comes from Google where Protocol Buffers have similar default-value behavior. I think Go would be better if it had made some different decisions in this area.
If you don't have a zero value, a programmer has to pick one. What are they going to pick? Probably what the language picks for you, "int n = 0;", 'string foo = "";', etc. For a language, it doesn't really matter which side you pick (force programmers to select a value, or auto-assign one). For network protocols, defining empty is an important optimization -- if the client and the server are guaranteed to agree, you don't have to send the value. This is especially important where the client and the server aren't released at exactly the same time; the server may have a new field in the Request type that the client doesn't fill in. With a predefined zero value, it doesn't matter. (You can always add fields to your message to get the same effect, if you actually care. I've never seen anyone do this in any API, including ones that use serialization that doesn't have the concept of zero values. It's why Javascript has the ?. operator!)
Finally, Go came out in the proto2 era, which did have the concept of set and unset fields (and let the proto file declare arbitrary default values). Honestly, I wrote a ton of Go involving protos at Google, and never saw proto3 until after I left Google.
Most languages without null have an optional type which is used exactly for that. In these languages, this means the None value exists when you have optional things, but that, then, the compiler forces you to check whether your value is set when you want to use it. Serialization libraries then get the choice to handle these optional values as they wish, which can be not to send a field.
It's one of those things that may be hard to think about from an external pov, but it works just fine.
From users of these languages, proto2 was okayish, and proto3 was a massive regression. Another thing that's missing in protobuf is the ability to define union types. That's one frequently asked feature from typed functional languages for serialization protocols.
func foo(maybeInt *int) {
if maybeInt == nil { panic("not so optional!!!!") }
...
}
func foo(maybeInt Optional<int>) {
switch maybeInt {
case None:
panic("not so optional!!!!!")
...
}
}
For protocols,an Optional type just pushes the problem one level down. Is the optional value "None" because the client didn't know the field existed, or because they explicitly set it to "None"? You can't tell.
I think rather than going 3 levels deep, it's easier to just define the default values and not distinguish between these three cases. If you want an Optional value, you can make yours as complex as you wish:
message Optional { int value = 1; bool empty_because_the_user_said_so = 2; bool client_has_version_of_proto_with_this_field = 3; }
Now if you get (0, false, false), you know that's because the client is outdated. If you get (0, false, true), you know that's because the user didn't feel like sending a value. And if you get (0, true, true), you know the user wanted 0. (Of course, there are all the other cases that you have to handle -- what about (1, false, true), or (1, true, false)?)
I think you'll find that nobody but programming language purists want this feature. If your message is:
message FooRequest { int foos_to_return = 1; }
You do the right thing regardless of whether the 0 in foos_to_return is what the user wanted, something the user forgot to set, or the user has an old version of FooRequest ("message FooRequest{}").*
They wouldn't really be different if a pointer were only used to express optionality.
However, in go, that's not the case. Pointers are used to also influence whether a receiver can mutate itself (pointer receivers for methods), to influence memory allocation, and as an implementation detail for interfaces.
If I have "func makeRequest(client *http.Client)", how can I know if the function will handle a nil client (perhaps using a default one), or if the client expects me to pass in a client, and just uses a pointer because idiomatically '*http.Client' is passed around as a pointer?
The answer is, I can't know. However, if we have what rust has, which is Box and Option as two different things, we can get 'fn makeRequest(client: Option<Client> | Box<Client> | Option<Box<Client>>)'. We've made it so the type system can express whether something is optional, and separately whether something is on the heap.
In go, those two things are conflated.
Similarly, rust has 'mut' as a separate thing from 'Option', which is another case where the type-system can express something the go pointer also is sorta used for.
In practice, I think there's a clear difference. Most of the pointers I see in go (like pointer receivers on methods) are not pointers because they're expressing "this might be nil", they're pointers because the language pushes you to use pointers in several other cases too.*
C/C++ implementations generally have a [[nonnull]] attribute to that effect.
One requires you to handle the possibly-nil case in order to access the pointer, the other expects you to remember to check every time.
The following is completely legal, and results in a runtime error:
func foo(maybeInt *int) {
x := *maybeInt // runtime panic
}
func foo(maybeInt Optional<int>) {
x := *maybeInt // compile error: need to unwrap maybeInt
}Also since Go supports nil receivers, it would have to require that every "pointer method" be checked for nil. Which technically they should, but...
For one, the compiler can force you to check for None. Trying to use an Option<T> as a T is a compile-time error, you have to write the pattern match to use the Some case.
For another, and this is the big one, you can write a function which takes a T, and you can't pass it an Option<T>. The compiler can statically confirm if your variable has already been "nil checked".
I used to write a lot of Java code, and since you can't know for sure that null-checking has been performed (among other reasons, that might change with a new code path) you just kinda sprinkle it everywhere. And still forget and get runtime errors.
func foo(maybeInt Optional<int>) {
switch maybeInt {
case None:
panic("not so optional!!!!!")
...
}
}
While I'd be happy to discuss about my experience using these alternatives and how it's practical beyond being a purist, it doesn't really belong to a discussion about Go.
Even (modern) C/C++ handles this aspect of memory safety better: There is no default value, and reading from an uninitialized value is a compile time error (usually, because C/C++ have baggage).
This was one of the significant changes from proto2 to proto3.
This was also met with much opposition internally, and recently changed as of v3.12 (released last year).
https://github.com/protocolbuffers/protobuf/blob/master/docs...
https://github.com/protocolbuffers/protobuf/releases/tag/v3....
In my horrible opinion, we shouldn’t ditch null. We must introduce null flavors (subclasses) instead and fix our formats to support these. One null for no value, one for not yet initialized, one null for unset, one for delete, one for type’s own empty value, one for non-single aggregation (think of selecting few rows in a table and a detail field that shows either a common value, or a “<multiple values>” stub - this is it), one for SQL NULL, one for a pointer, one for non-applicable, similar to SQL. Oh, and one for not-there-yet, for async-await (a Promise in modern terms). These nulls should be enough for everyone, but we may standardize few more with time. Seriously, we have three code paths: normal, erroneous and asynchronous. Why not have a hierarchy of values for each?
Semantically all nulls must be equal to just “null” but not instanceof null(<other_flavor>).
Edit: thinking some more, I would add null for intentionally unspecified by data holder (like I don’t share my number, period), null for no access rights or more generic null for “will not fetch it in this case”. Like http error codes, but for data fields.
Here is different proposal. Let's allow people to define their own types of missing values. We'll call it Nullable<T> or Maybe<U>.
It's better you develop one for you and that suits you, rather than just a set of null-likes that are similar in meaning, but different in semantics.
Such a system can be done, but it is probably super ugly and confusing. The usual answer instead is: exceptions, i.e., instead of instanciating an error object, throw an exception (well: you do instanciate an error object here...). That works, but if overdone, you get programming my exception, e.g., when normal error conditions (like 'cannot open file') are mapped to exceptions instead of return values.
The usual answer to that problem then is to use a special generic error class that you specialise for your type, the simplest of which is 'Optional' from which you can derive 'Optional<MyType>'. You can define your own generic type 'Error<MyType>', with info about the error, of course. I think (please correct me if I am wrong), this is currently the state of the art of doing error handling. It's where Rust and Haskell and many other languages are. I've seen nothing more elegant so far -- and it is an ugly problem.
[0] https://doc.rust-lang.org/std/result/ https://caml.inria.fr/pub/docs/manual-ocaml/libref/Result.ht... https://hackage.haskell.org/package/base-4.15.0.0/docs/Data-...
The way this is addressed in Google's public APIs is with the use of a "field mask"[1]. You provide a list of dotted paths to the fields you want to update. I'm not sure if that serves as an indictment of the design decisions made in protobuf, or if it's just one less bad tradeoff among several bad ones.
[1]: https://github.com/protocolbuffers/protobuf/blob/e9360dfa53f...
To me this looks like a discussion about syntax and ergonomics. Go provides the same mechanisms:
* for safe dereferencing, use "val, ok := *valRef"
* for potentially-unsafe dereferencing, use "val := *valRef"
Every language in that list has equivalent mechanisms. In Rust you can use one of these methods: https://doc.rust-lang.org/std/option/enum.Option.html or pattern matching. That's a whole lot to pick from.
But Rust also makes it more complicated to reason about your memory, see for example https://doc.rust-lang.org/std/option/#representation
So, given Go's design tenets, using pointers makes a lot of sense to me. It is easy to reason about them in terms of memory and resource consumption, there are only a few ways they can be used, pass-by-value semantics further reduces the centrality of them, and they don't require a JIT compiler to be efficient.
Edit: Nevermind, I forgot this is only of T is a reference. Otherwise it's layed out like a normal enum.
Relatedly, in practice, a lot of Rust code I've worked with is littered with unwrap() calls.
Not that "maybe it uses an extra byte, maybe it doesn't" is going to matter in 99% of situations.
Interesting side-point re. language comparisons I noticed recently -- Java is often benchmarked together with compiled languages, although I would say it's only half-compiled (to bytecode, not to machine code). That's
But yeah: I once heard Go described as "a C fan-fiction". It was designed by and for people who use and like C, and just wanted some niceties built on top of it for their usecase of writing distributed systems (garbage collection, strings, easy networking and multithreading, easy cross-platform builds, very basic polymorphism). It was not designed by people who were interested in stepping back and re-evaluating the big picture beyond those practical necessities
I see it as C++ (literally, "C improved"), for the modern world (instead of the 80s). For all the good and bad that implies.
It's more of an alt-java than a C fan-fiction.
Most of the stuff people are interested in in C, Go dropped. However while Go clearly disagreed with Java on the specific means and most important bits the core goals and aesthetics are extremely strongly reminiscent of Java's.
That's a consequence of the "always mutable" model whereby the responsibility of initialization can be shared by the object constructor and the caller.
But there are many cases where it's very intuitive for the caller to set up an object, especially in the objects as actors model.
I think to make that work, you'd want to track the evolution of the object's state, especially noting when all fields have defined values.
I've seen all these approaches in Rust.
It's not though. An "always mutable" model can require that objects be fully initialised up-front. Rust does exactly that.
The reality is that you don't know where your nil-dereference bugs exist because your language doesn't detect and force you to eliminate them.
Which means you can check after a few years how many nil-pointer exceptions you got in your actual system. In my experience that numbers is really low.
Not a great experience for your users though. And definitely not great for developers who get paged to handle these production incidents that could have been caught at compile time.
Null made since in the early days of C, an option type back then would have been prohibitively expensive. Today, not so, any language that can afford garbage collection, the cost of an option type is lost in the noise, and modern compilers can often optimize the cost away entirely. (though that would hurt Go's fast compile times a little bit)
Although I would agree it was possible to design the language much better to avoid Null, most buffer overflows etc.., C was not designed to be good from PL standpoint. It was designed to write something and then evolved.
Of course not. Option types are as old as C, and they don't cost anything more at runtime than a regular pointer, or a union tag, costs.
I suppose not, if you are considering "we" as "the set of people that designed languages like Go or justify the design decisions made by such group".
If you are considering "we" as humanity, in general, I'd say that no, "we" actually learned a lot and "we" already know better. But you still have some building rockets to try to prove the earth is flat, unfortunately.
And I'd say it fails at that goal, judging by the number of gotchas in fundamental concepts of the language like slices or interfaces.
A surprising behavior of slices that had discussions on HN, and something I discovered in my own code as well: https://news.ycombinator.com/item?id=9555472
And of course this gem was on HN just yesterday: https://news.ycombinator.com/item?id=26631116
This was never a goal of go. Go was conceived out of frustration with c++. They wanted to reduce language complexity and build times (among other things). For me personally it's C without the hassle.