Things Go needs more than generics
betterprogramming.pub
betterprogramming.pub
What Go actually needs: Nil-Safe Types! [1]
Programmers can work around verbose error handling(3) and lack of enums(1), but forget to check for nil before using a pointer... CRASH! And that's something the compiler doesn't warn about.
[1]: https://wakatime.com/blog/48-go-desperately-needs-nil-safe-t...
Like, I'm not trying to be cute here, but the point of having a "non-nullable type" is that you can get an error message when you use a nullable type in its place. However, if you're not talking about non-nullable types and just about whether the compiler knows a value is nil, well, the compiler knows that.
The safety is contagious, preventing crashes from nil pointer dereferences at runtime. For ex: 99% of your program is functions accepting only non-nullable types then 99% of your program is guaranteed to not crash from nil pointer deref.
With Go, you have to check for nil in every function even if all your callers already check for nil. That's because Go can't declare a function's arg as non-nullable type.
I have no idea what you mean.
func foo(a int, b someStruct, c *someStruct)
Now, it is true that there are multiple reasons for using a pointer aside from supporting nil (ability to modify, optimization not to copy a huge object).
Your claims are correct, but it feels like you're missing the point they are (ineffectively) making.
You make it seem like value types solve this problem, but they don't. C also has the same thing you're talking about, but it still has nullability problems.
I think other comments are not even asking for anything complex. Something akin to C++'s references (which cannot be null) would already be a step forward.
Are you sure? This would only work in a world with a single goroutine.
As do TypeScript, and Kotlin, and Dart, and Python (with MyPy), and C#, and Swift...
It's honestly inexcusable that Go, as a relatively new statically-typed language, doesn't have strict null-checks
This isn't to say it's useless, just that it's not as useful as it _could_ be. I've seen codebases in all three languages where there have been real production issues due to null values, even though the type decls say otherwise. Kotlin probably does the best job here but using any JVM library introduces potential (K)NPEs.
Languages like Rust, Haskell or ML do not have this issue since they do not need to support such a legacy layer.
Still, typescript equips you to catch these errors, even if you can technically circumvent it. In practice it can be nearly bullet-proof if you follow good practices.
- !
- as x
- @ts-ignore
If you're using either a library that wasn't written in pure TS (maybe JS or JS with .d.ts) or interacting with some unconverted JS from your own codebase, you can easily pass a null through entirely by accident. The problem really stems from the JS end of things, but 9 times out of 10 you're going to be touching JS at _some_ level when using TS so I think it's fair to point out this gap.
You can override TypeScript's checks for anything, not just nulls, and yet people still get tons of benefit from it. Same goes with many of Rust's safety checks in unsafe { } blocks.
Additionally, languages like C# and typescript have to be a little more flexible because they added this feature after the fact. Go had plenty of opportunity to do it up front and not require any compromises, and it chose not to.
E.g. nullable types are not options - you can't "map" them (you can't invoke Select, Where etc). It's easy to "get value or default" (via operator `??`), but you can't do the other, equally frequent thing, of "apply transformation on value if not-null". Or... well, you can, but here's the syntax:
var newVal = val == null ? (resultType?) null : compute(val.Value);
var newVal = val.Select(compute);Of course that doesn't fix all the other issues with null handling in C# (inconsistency between null references and null value types, inability to nest options, ...)
Now I consider nil safety as an essential feature of any modern language.
While I like Rust, it would be good if people actually did their research while crediting Rust for things it didn't implement.
* invent
(Other than this minor typo, I'm not sure why you're being downvoted. It's crazy that people in 2021 think non-nullable types are novel/"crazy". Why is nullability the default in most people's brains?)
More generally, I don't think many programmers get to see the better way because they're not exposed to it. I can rhapsodise about Rust, but I doubt my company is going to buy into it because they already picked Go.
Go is not a language that "just happened" (what could be said of JS and PHP). Go is designed. Designed by heavyweight language designers (from Wikipedia): Robert Griesemer, Rob Pike and Ken Thompson.
These people knew of Haskell, type systems, and the merits of type safety. I would expect them to know of "billion dollar mistake[1]" that is null. But sadly Go still carries on with the mistake.
I too care more about null-safety and proper sum types (in combination with nice switch/match statements and/or pattern matching) than generics. In the Elm language I found an experience that not having generics is perfectly okay (just a little annoying sometimes).
I find "not null safe languages" not okay nowadays, and I hold this opinion since before Go's first appearance (2009). I really wonder how the designers came to this decision.
I'm afraid this mistake can never be fixed, as null checks are already idiomatic Go. Java also could not fix it (which may be one of the main reasons behind Kotlin). Maybe the best thing we can hope for is Kotlin kind of language for Go (question marks after types to indicate nullability). It's just sad.
[1]: https://www.infoq.com/presentations/Null-References-The-Bill...
What makes you think that they didn’t know about it, rather than that they did know but decided they weren’t interested in the trade offs for this particular new language.
C# added some null-safety features despite having a 20-something year baggage of legacy code. While it might not be the perfect solution (these checks work only at compile time and you can enable them per-file), I find that they work great for new projects. You have to be extra careful at boundaries (interfaces with third-party libraries which have not added ? annotations yet, API calls, and so on), but they save a lot of headache inside your own code.
The question mark is the best thing if you have not build your language with null safety to start with. Please look at Elm for a good example of what real null safety looks like.
With Go they had to change to do the right thing from the start. I really wonder why they didnt.
I did not read the parent comment as "rust invented nun-nullables and is great", but as "rust, for example, removes the class of problems this way...". I think its neither helpful nor realistic to require every mention of a feature to be backed up by a proper research into who invented in when.
So worth giving it credit where credit is due for choosing to use something other languages showed were a good idea. But it's all design choices now, there's few major new language ideas in the mainstream, so while I agree it would be nice to see a little more awareness of the legacy, I wouldn't phrase it as confrontationally as "if people actually did their research" when most people only write short comments and replies that probably assume you also know the legacy.
I think here you mean that C-style nullable pointers should be “constrained to low-level programming” and high-level programs should always have option types and non-nullable pointers, that correct?
If that’s the case then I’d say “nay”… because it’s not that hard to have your cake and eat it: while Rust implements niche-value optimisation somewhat generically, nothing precludes special-casing optional pointers such that optional and non-optional pointers have exactly the same ABI.
And then if the language is memory-safe it probably gains in performances, because it doesn’t have to check non-nullable pointers for nulls before dereferencing them.
If so, yes, that makes complete sense. But if it's on the level of system languages, well, Rust is a perfectly capable low level system language, and constraints nulls only to where they matter.
At the end of the day, null is a value for pointers, and a system language does need pointers. But if you have a reasonable type system, not everything needs to be a pointer, and the type system is a compile-time feature, so it doesn't matter for your code target.
Yes pointers are there, for when they are really needed for interfacing with the hardware, dynamic datastructures and reference parameters can be dealt in more type safe ways.
There are simple, go-like solutions. There's a character for pointer, so why not a character for a non-nil pointer?
Tbf, this is simply solved by putting a generic optional in the stdlib.
In Rust, you're forced to match against the enum before you can look at the value, but in Go you even can't match. So unless it's implemented as a type cast, testing optionality would be just like testing against nil. The current internal "optional" package just panics, which is just as useful as a panic on dereferencing nil.
Give it an interface that allows you to attach behaviours for not null and null values.
[1] https://docs.oracle.com/en/java/javase/11/docs/api/java.base...
As a Rust security consultant, many bugs we find in Rust applications are crashes due to .unwrap()... it's easier to spot, but still essentially the same class of bug.
Isn't expect() supposed to cover that case?
expect("something bad we don't expect")
unwrap_or_else(|| panic!(...))
is clear and doesn't suffer from the naming mistake.
In Go some common types are forced to be nillable, and you can't express "this is never nil". In Rust, "never nil" is the default, even for slices and by-reference types, so right of the bat for the majority of types nullability disappears entirely. You can't make a mistake of `unwrap()`ing something that doesn't support unwrapping.
`unwrap()` is the laziest/worst way of handling optionals, but it's still better than Go's behavior. `unwrap()` is local and explicit, so you can find and review every potential failure point (unlike e.g. finding every use of a nil map in Go). And of course Rust has plenty of better, graceful ways of handling optionals, so you can also ban this method entirely with a lint.
It does yes. A fair number of other constructs can panic as well.
> I wonder if any codebases lint those away.
Clippy has a lint for indexing so probably.
For the general case, it's almost impossible unless you're working on very low-level software (embedded, probably kernel-rust eventually) e.g. `std` assumes allocations can't fail, so any allocation will show up as a panic path.
https://github.com/Technolution/rustig can actually uncover panic paths, but because of the above the results are quite noisy, and while it's possible to uncover bugs thanks to rustig it requires pretty ridiculous amounts of filtering.
The `for` loop is based on the Iterator trait. Iterators typically optimize better than indexing, and can't panic.
Yeah it did.
> Just traded it for something slightly different. ie, calling unwrap when no value exists causes the program to panic.
That betrays an absence of familiarity with option types.
First of all, an optional pointer is strictly more work, so you're not going to use an optional pointer when you don't have to, meaning most of your pointers will be non-optional and statically checked so.
This means when you do encounter an optional pointer, there's a reason for it. At which point you get to put your thinking hat on and wonder whether that's applicable to your situation:
* sometimes you don't care because it's a one-off or something and you just unwrap() and go on your merry way
* sometimes the pointer is set by construction but the typesystem is not expressive enough to understand that, usually by the second or third time you get around that code and don't remember why the unwrap's there you'll switch to `expect` or `unreachable!` in order to document your assumptions or logic (which is also helpful when those break and the code panics
* and most of the time there's a good reason why the pointer is nullable and it applies to your situation and you probably want to handle it properly and you do.
Plus `unwrap()` and friends are easily greppable so you can review them with little trouble, or flag them during review, or whatever.
In language with nullable pointers (and only that), you've got none of this.
In C# we were going from "every reference type can be null" to "only things marked with '?' can be null" with nullable reference types. When I started using Go it was nice to see that only something, that is explicitly created as pointer is nullabe, everything else can't be null / nil - and you can't get panic from nil referencing stuff
Well, it's the same in C# since 1.0 to some extent, except that people rarely used `struct` (value types) in C#. The ecosystem matters a lot, of course, but just at the language level, Go and C# structs have the same semantics (though in C# it's harder to get a reference to a struct).
It seems like one of those fundamental advancements in language design, like when computer science collectively decided goto should be deprecated in favor of flow-of-control expressions.
* val input: String? is a nullable String. It's basically String|null.
There's a third option, which is String!, platform types: since Kotlin has interop with Java, Kotlin will err on trying to be practical for you when calling Java code: it will (rightly so) assume that it is a non-nullable String, but still warn you that it doesn't have the necessary info to infer nullability/non-nullability, so it could technically be null. It's up to you to decide if you want to null-check it. This can be solved in two ways:
- Update your java code to include @Nullable/@NonNull annotations. You should already be doing that anyways, especially if you have control over it. - Don't call Java code/wrap it in Kotlin wrappers. Kotlin code does not have this type inference issue.
(Bonus point for Java that has introduced Option types, that can still be null)
String? :: Option
String :: Option.Some
null :: Option.NoneThis interacts terribly with generics, since it means that only one side of the generic boundary can "own" the null value at any one time. To simplify a case where this can cause issues:
class Loadable<T>(
// null until value is loaded
var valueIfLoaded: T?,
)
// returns null if user is not found
fun getUser(id: String): Loadable<User?>
The "correct thing to do here would be to add a `T: Any` bound on Loadable, which prevents T from itself being a nullable type. That would notify the author of getUser that they need to box the User?, so that the cases are kept separate:
data class Box<T>(val value: T)
class Loadable<T>(
// null until value is loaded
var valueIfLoaded: T?,
)
// returns null if user is not found
fun getUser(id: String): Loadable<Box<T?>>
Nullable types are a feature of the type system and require language support, but the benefit is that you don't touch typical programming patterns
val foo: T? = x()
if(foo == null) { return ... }
// foo is now T, optionality gone
val foo: Option<T> = x()
// following must be wrapped and sprinkled in .map, .flatMap, .orElse
Really? I haven't used Java in a long time, but in the languages where I've used Optional types, you'd always check for presence once and then extract the value before further use.
This equation makes sense for some mental models, but isn't quite the same as Option. In particular, Option[T] has the nice property that you can map over its inner type in a naive fashion `forall S. (T -> S) -> (Option[T] -> Option[S])` whereas the coalescing version above needs an additional assumption that `S` is non-nullable.
forall S not null. (T -> S) -> (T? -> S?)
That said, if one ever finds themselves seriously using a nested Option, it's probably time to write a new enum for that use case.
I like Swift’s approach personally, which is that there’s an Option type, but loads of syntax sugar to make working with it easy (just append a “?” after the type to flag it as optional, “if let”, optional chaining with “?.”, “?()”, etc etc.)
Option types require generics, which Go doesn't (currently) have and may or may not be enthusiastically embraced by the community.
Option types must also be baked into the language, standard library and culture from the beginning in order to be ergonomic; otherwise you're constantly having to wrap and unwrap them whenever you interact with code that was written before the type was introduced. Which is most the code you interact with.
Finally, my own experience has been that Option types generally make things more rather than less complicated when you introduce them to a language that already has null. You'll need to make a decision on whether Some(null) is an allowed value, both options will introduce language warts.
Kotlin's approach, on the other hand, does not require generics to work, and interacts very well with the JVM's existing libraries and its existing type system. It's not going to butter everyone's toast. You can't do anything monadic with Kotlin's approach, for example, but I don't get the impression that the Go community is just desperate for monads.
Why would a built-in option type require more genericity than the built-in array and map types?
The other big distinction between the Kotlin approach and optional types is that the Kotlin approach introduces no new run-time types. Null safety is checked statically, and then you're done. That's a big part of why it plays nicer with an existing standard library. It also means, though, that you introduce no extra run-time overhead, which I would assume is something that's considered pretty desirable to gophers.
void foo(int? x) {}
void main() {
foo(5);
}
fn foo(x: Option<i32>) {}
fn main() {
foo(5);
}
A couple things I have tried:
- hope default values align with your business logic, eg an empty string isn't a valid name and 0 isn't a valid age.
- for partial updates, populate the existing values before unmarshalling, then unmarshal on top. Missing fields in the json won't overwrite the existing values
- unmarshall into a map[string]interface{}, which gives you the semantics you want.
My own cranky opinion is that, in the 21st century, a static type system that can't even reliably distinguish between something and nothing isn't much of a type system at all. Say what you will about dynamic type systems, but it must be acknowledged that they do a much better job of maintaining a firm and logically consistent, if not statically verifiable, grasp of the most basic ontological distinction that the universe has to offer.
There's always the option to wrap the atomic value in a struct of some sort. But, without generics, that's going to feel more like gRPC's awkward, verbose way of doing it than the ML family's Option<T> type. That approach is (mostly) tolerable in a datagram format, because I/O is going to be a hot mess no matter what you do, anyway, but I'd hate to have to do it with the types I'm using in my actual code.
I'm generally fairly disdainful of the, "every language must cargo cult everything functional languages do," thing, but it is interesting to observe that generics and proper algebraic types would cover all of these use cases - and more - cleanly and without having to add a specific language feature to cover each one. Which has me wondering, if the goal of Go was to create a maximally simple and ergonomic imperative language, did they actually achieve that, or did they follow a greedy search into a local maximum?
If a function says it returns a struct and an error value, then that function is going to return things into a memory chunk sufficient to hold that struct and error value. "sizeof" is a well-defined operation in Go. Everything has a precise size the compiler uses. Slices may look dynamic, but they're actually a three-word structure for size, capacity, and pointer; the "dynamic size" happens behind that pointer. Maps may look dynamic, but they're a fixed struct as well under the hood. Channels may look like they could be dynamic, but they have fixed sizes as well for their internal components. Go is a value-based language, not a reference-based language.
The same thing happens when you declare a struct value in a function. Memory for it is allocated immediately.
"Initialize with empty value" is not a solution to any sort of type issue and has nothing to do with nil; "initialize with empty value" is a solution to C's uninitialized value problem. Unlike C, which simply grabs some RAM and gives it to you, and leaves it to you to figure out what to do with the garbage values in it, Go guarantees that all fresh values you receive are fully initialized to a well-defined "zero value".
In Go, when you say you have a struct, you do, right then and there, and so, it needs a value. You don't have an "undefined" value, because Go is too low level for that. For Go to have an undefined value for a struct isn't something it could just bodge on, it isn't something that was just an "oversight", it would actually be a fundamental overhaul to the language. Go would have to be adjoining values to the structs you declare, making it harder for you to know how much memory they take, or it would have to completely shift to a reference-based language, or it would have to do something else like that not in keeping with the nature of the language.
Rust, for instance, is doing more work than you may realize when you "Option" something. Think about what the memory representation of Option<byte> is. Without more information from the user, you can't use any value of the byte as the undefined value, because all 0-255 values are valid byte values. You have to stick it somewhere else. Rust, and some other languages, often do some magic to turn your byte into a 16-bit int instead and pack the invalidity into there. Having an array of Option<...>s is a non-trivial operation for the compiler, especially to do it maximally efficiently. Go doesn't do this kind of thing. The struct that is declared is what is in memory.
I welcome you to think that's still a problem in the language, but I hope this at least makes why Go works this way more comprehensible.
> Having an array of Option<...>s is a non-trivial operation for the compiler
This is just not true. Arrays are always a number of values laid out in memory, exactly like in C. There is no magic for an array of options. It works the same as an array of any other value: you put N of them in memory, one after the other.
Sum types in general usually have compiler magic associated with them, because the common case of options or sums between various integers can get good results with the compiler being smart enough to pack the "None" option somewhere clever. As the size of the thing being used as a sum type goes up that amortizes to being less important. The naive way of using some integer as a tag and then having a chunk of memory large enough to store the largest value in the sum type can get very inefficient for small "largest values", especially if you have to round to a full 64-bit machine word for some reason.
Also, to be abundantly clear, I think this is all a good thing. It is intrinsically part of the value of a sum type in a language that you're not forced to think about the modestly complicated memory layout such things entail. It is good that compilers have some special cases for when you're summing on small values.
Seems more likely you'd limit your enums to 256 entries and always use a byte than require a word by default, no?
niche-value optimisation is a much more complicated affair so it's unlikely as a baseline indeed.
Personally I prefer how Protocol buffers handles not present strings compared to C. If you stick to std::string, C++ isn't bad either.
I don't see how making types non-nullable solves the problem, doesn't it just create a new problem? When I load the string from somewhere outside of the program, if it is not present, I would either have to check for that condition otherwise I will crash, otherwise I would assign it some magic value and have to check everywhere "if not magic value".
otoh, if the strings are not external then there is little danger of things ever being null no?
This is a very important programming pattern that should be used at every available opportunity, and probably one of the biggest programming failures that is rarely talked about in programming. Dynamically-typed languages encourage this style more than statically-typed languages but it can be done in either one. You need to do as much of your validation as close as you can to the time that data enters your system. If you don't, instead of validation living at your edge, it gets smeared through the entire program, and it will be done incorrectly when that happens.
(If you have a deeper layer that has more restrictions on the validity, then the additional validation should be done on that deeper layer. This can be applied recursively within a program if it is big enough to have multiple domains. But you always want edge validation.)
Architectures that fail to do this inevitably become a mess on the inside. Functions develop that are passed "strings" but they don't know if they're validated strings, or decoded strings, or what. Where things get validated and decoded becomes incredibly complicated. Functions start growing options describing what's being passed, but then the functions calling those functions end up eventually being wrong themselves. This eventually metastasizes into the sort of code that nobody can change because every attempt to change something screws up some delicately balanced code path. The only solution to this is to start over and do this edge validation I'm talking about, so that the entire rest of the code base just stops having to worry about it.
This also goes hand-in-hand with the idea that you should decode data at the edge, and pass around data internally only in its natural format, and encode data only at the edge as it leaves. If you're in the "middle" of a program and suddenly there's code to URL decode a string, there's an architectural problem in that code. (This code is likely to become code in the future to conditionally decode the string, or much worse, guess at whether the string needs to be decoded, which is getting perilously close to a You've Already Lost situation.) If you come to a deep enough understanding of what it means you can start to see that validation and decoding into a canonical internal format, and the encoding on the way out from the canonical internal format, are the same thing.
I know this is a common question since I also had it myself at the beginning, but see the strong types not as the assertion that the world will never have a nil string in it, because that's obviously an impossible assertion, but as a statement that any calling code is going to have to deal with what happens when there's a nil string (or whatever other invalid input), because I, this strongly typed library, am not going to deal with it. This statement is also almost always correct, too, because the library lacks the context to know how to handle things it can't handle. You shouldn't ask libraries to handle things they don't know how to handle, because, well, they don't know how to handle them.
But I sort of agree that excluding nulls is only part of the solution, because existing isn't the only requirement that values have to meet. I often need strings to be non-empty or have a minimum length.
Many modern type systems allow you to define new types that conform to an existing type's interface. But it's often a rather convoluted affair.
If the type is non-nullable then the compiler will refuse the program if you have not dotted your is and crossed your ts. So you'd have to check your expectations at the edge, and if the expectation is "that is indeed optional" then it's optional internally and the compiler will ensure that is acknowledged at every place it's used.
> otoh, if the strings are not external then there is little danger of things ever being null no?
It depends. Lots of things are internally nullable. If I look an account by a key that's been provided externally, the account may or may not exist, that's an optional.
However it's true that most of the internal values will be non-nullable, and then the advantage of non-nullable types is that is checked and validated by the compiler, so it avoids misuses and misunderstandings.
When you only have nullable types, then the only thing you can do is assume, everywhere, and pray that you're right.
Universal Nil: no. we need less nil not more. Nil-safe types would be a better answer.
Error-checking: If you're not going to do anything with the error, then have an editor snippet for the ? boilerplate. Better still, start handling the error. The boilerplate is a Go anti-pattern.
I think we do need more clarity about the relationship between slices and arrays: it's not clear at the moment when two slices share a backing array or not, and therefore whether changes to one slice will change the other.
I'm kinda looking forward to generics, but also dreading it. Most newbie gophers don't understand how powerful duck-typed interfaces are and what they can do with them, and I have a feeling generics will be used for all sorts of situations that could be better handled with interfaces.
> Better still, start handling the error.
Most of the time, you can do this only two-three levels higher on the call stack, and even here it's most likely to be wrapping up of "return nil, fmt.Errorf("frobbing bar: %w", err)" sort.
And yes, I am too excited and scared of the generics. I'd like to see generic map- and channel-related algorithms and patterns, and sort.Interface is an ugly hack, even if an ingenious one.
hmm. I don't have the problem you're describing, because I generally declare the return struct at the start of the function and then return that. E.g.
func something() (mystruct,error){
result := mystruct{}
if badthing() then {
return result, fmt.Errorf("bad thing happened: %w",err)
}
//do something to result before returning it
return result
}
Perhaps gofmt could even automatically replace it?
The problem with this is that, in a larger function, I have to read all the way up to confirm if result is an empty value or something else. I think returning myStruct{} is much better, and having a way to easily return a default myStruct would be nice. Calling that "nil" seems to be a bad idea to me, as others are pointing out.
> I must admit I don't understand how the proposal works fully. Would `x==nil` return true for an int with value 0? Both answers to this seem wrong to me.
I think it would return false in that case. There is already precedent for this in Go, in fact, with arrays/slices. Even though slices themselves are value types (they wrap a pointer) they can be nil, but []someStruct{} == nil is false.
If in the proposal (x==nil) == false then we get the ridiculous position where:
var x int
x=nil
if x==nil {
//this never happens despite having just set x=nil
}
Though the other case, of returning true, also seems ridiculous, because nil and 0 are not the same thing at all.
That's why I actually would prefer the "_" syntax:
var x int
x = _
_ = 42
if x == _ {
// this happens even though you've assigned 42 to _, yes
}Anyway, I do agree that using `nil` as a base case for all types would introduce more confusion. What I think could be useful would be a `default` keyword for this:
var x int
x = default
if x == default {
this is true
}
Though, of course, most coders would leave out the second step here since the compiler initialises all variables with their type's default value ;)
return default, default, default, errPart One: Endless Error Handling https://jesseduffield.com/Gos-Shortcomings-1/
That blog also covers a lot of Go's wonderful defects advertised as 'features'.
The latter tends to make the eyes glaze over in tedium after some time and due to the signal/noise ratio one misses real bugs in the error handling.
I like that Go forces at least some kind of error handling.
The "signal/noise ratio" problem fades after a while - you start seeing "if err != nil {...}" as a single statement after a while, a steady rhythm in the code. You notice when it's not there, or when it's doing something different.
I don't understand this point but I hear it raised often. At best I'm lukewarm on Go's error handling. Having if err != nil feels the same to me ergonomically as a try / catch.
Am I missing something by thinking that?
Then there's exceptional exceptions which should be not crash near the "edge" of the program and need to be caught and something done with, so you add those to the program.
What you wind up with using that approach is only exception handling being written where you need it, and the vast bulk of the code is understood to always crash and get handled in some inner loop, or even some external supervisor.
Go feels like it was a reaction to (probably early-2000s) Java programming patterns where at every level Java programs were catching and decorating and rethrowing their own exceptions. Neither of those approaches really are helpful though because you litter your code with all kinds of exception handling that nobody really cares that much about, and under both systems you can crash instead of retrying and all the useless error handling code you've written everywhere doesn't help you see where you've made that error. The 2010 erlang crash-first programming mentality works way better imo rather than error checking everywhere.
And Go out of the box makes it difficult to get stack traces, so you should really use an error library at least.
That really is the scariest part about it. I really hope the Go community will take this issue to hearth and push back against codebases with unnecessary generics. Otherwise the whole ecosystem will suffer from it.
Duck-typed interfaces are kind of broken in Go, though. I can use an Impl anywhere I can use an Abstract, but if I have a function that takes a slice of Abstract, I can't pass it a slice of Impl, because it's the wrong type.
type Abstract interface {
value() string
}
type Impl struct {
str string
}
func (impl Impl) value() string {
return impl.str
}
func ShowAbstracts(arr []Abstract) {
for _, a := range arr {
fmt.Println(a.value())
}
}
func main() {
arr := []Impl{Impl{"a"}, Impl{"b"}}
ShowAbstracts(arr) // This doesn't work, because arr is []Impl, not []Abstract
}
Slices are backed by arrays, not actual arrays - copying a 100K slice into another slice isn't as intensive as it sounds.
I'm not sure I see this as "interfaces are broken in Go", or something that requires generics to solve? (I do realise that the language itself uses generics to implement the copy keyword for this).
Generics fix this handily, because I can write:
func ShowAbstracts[T Abstract](arr []T) {
for _, a := range arr {
fmt.Println(a.value())
}
}
The problem is that ShowAbstracts could just modify the array and put an instance of Abstract there that _isn't_ an Impl, thus violating the original constraint of arr containing only instances of Impl.
Once again mutability/invariance raises its ugly head!
Everyone needs optional values, due to Go's poor design the closest to this is to use nil.
> I have a feeling generics will be used for all sorts of situations that could be better handled with interfaces.
Very much agree. I'll also extend "interfaces" to include closures.
2. Nil - no.
> The way I see it, there is a simple solution staring us in the face — make nil a placeholder for the zero value for any type. [...] Sure, it might be a little odd to see nil used in place of 0 for integer values, but I think this is a small price to pay [...]
So is writing `return MyLongTypeName{}, errors.New("...")`. The power of Go is its explicitness. It's a slippery road to introduce an unintended behavior. The existing use cases for nil are extremely clear. There is no need to change that. I agree that it's annoying to write `return MyLongTypeName{}, errors.New("...")`, but it gives you a chance to think about a better name of a struct and normalize errors. Suddenly, an ugly looking line of code becomes short and nice `return Type{}, ErrInvalidOp`. Maybe Go would benefit from a reserved keyword that would represent a default value of a type, eg
return default, ErrInvalidOp
var x int = default
var y string = default
return default, ErrInvalidOp
...
return 0, ErrInvalidOp
// was it intentional to write 0 here? can it be replaced with default? is there a difference between 0 and default?
What if you have more than two return values?
func Foo(x int) (int, error, error)
y := Foo(x)?
I feel it’d be easier to have “sealed interfaces”: Go already has type switches, so the only missing pieces are defining sealing and adding match completeness support to type switches.
> The power of Go is its explicitness.
And yet it’s got plenty of implicitness. Though I guess you’re fight in that most of them are footguns?
> 3. Concise error handling - no
Then you don’t use concise error handling? Seems pretty easy.
right, obviously.
Having a ? operator that avoids the `if err { return nil, err }` would be awesome.
The case you are representing is a clear "anti-pattern" in Go IMHO. Why should a function return two errors?
In any case, the ? operator could simply not be applied in this case.
> the ? operator could simply not be applied in this case.
No. Go doesn't want to have such operators exist at all, because they are not simple. Go wants to find the most common denominator that would elegantly solve an issue from all angles in such a way so that the authors don't have to write pages of specification, but most importantly, so that code readers don't have to read specification when glancing through code.
But in principle with generics and possibly some other additions you could provide much more useful checked exception types: ones that are applicable to your situation and would allow you to bubble up more precise checked exceptions.
You need a wrapper for 0 exceptions, a wrapper for 1, a wrapper for 2, …
Kotlin did the right thing by not implementing checked exceptions for now.
Yeah and Java has pretty thoroughly poisoned that well so in every discussion of the concept what comes up is mostly Java’s implementation and then it’s dead on arrival.
And then you’ve got Swift which uses an exception-ish syntax for results, but they’re not even remotely exceptions, and functions must say that they are faillible (or failure-transparent) but then can’t say what their failures are, so it always feels like the worst of both worlds.
Though it makes more sense when you understand that it’s dual to the old `NSError*` out-parameter system.
You could have a `?` macro which only handles the standard case of two values. If you have more than one, then likely you need to handle it in a specific way anyway.
Having `?` does not mean the previous way of checking error doesn't work anymore.
Consider this situation(it may sound familiar): you get paged at 4am for 500s from a service, check the logs, see 'file does not exist'. go doesn't attach stacktraces to errors by default. go doesn't enforce that you wrap errors with human context by default. How do you debug this? Pray you have metrics or extra logging enabled by log level that can give some degree of observability.
This error could have been 'opening config: file does not exist' or 'initializing storage: opening WAL: file does not exist' or even just decorated with a stacktrace. Any of those are immediately actionable.
Now, if go decided to make error wrapping a first class citizen with a fancy '?' operator, I'd be excited. However, I doubt that will happen because go is definitely not rust-like in its design.
if err != nil {
return fmt.Errorf("What I was trying to do: %w")
}
[1]: https://go.dev/blog/go1.13-errors [2]: https://github.com/pkg/errors
Second of all, nothing prevents `x ?= Foo();` from implicitly doing `if x,err := Foo(); err != nil { return fmt.Errorf("Error in abc.go:53: %w", err)}`
Searching our prod code, "naked" if-err-return-err showed up in about 5% of error handling cases, the rest did something more (attempted a fallback, added context, logged something, metric maybe, etc).
I am not a fan of manually wrapping errors because it seems inferior to stack trace.
Also, I hate that Errors in Go are mostly just random strings, super hard at a high level to do anything intelligent.
If anything, these proposals have been even more controversial than generics. The "try" proposal in particular, was probably the mildest proposal of them all, but it garnered a lot of criticism which amounted to "Go error handling is perfect as it is".
Personally speaking, I vehemently disagree with the criticism. I think that error handling go is a meteor-crashing-into-earth level disaster and anyone happy with that must be completely out of their minds. But there does seem to be sizable (or at least very vocal) group of Go users who believe this boilerplate help understanding the code better, and I don't think we can reach common ground.
It makes sense. Due to its design, Go naturally attracts people who we believe in can call "Imperative Explicitness Maximalism". This is the belief that the code is easier to read the closer the code is to to description to the way your machine executes code[4].
The people who want to reduce boilerplate as much as possible may be motivated by various beliefs, but I think the most credible belief is that imperative explicitness hides the _purpose_ of your code behind nitty-gritty implementation details. From my perspective, in cases where the implementation is trivial (like error handling), you're adding nothing to understanding the implementation, while completely hiding the purpose and — at the same time — significantly increasing the signal-to-noise ratio in your code. So you get zero readability gains at the price of two significant readability losses.
---
[1] https://github.com/golang/go/issues/40432 [2] https://go.googlesource.com/proposal/+/master/design/go2draf... [3] https://github.com/golang/go/issues/32437 [4] Of course, even this group has a limit. They tend to view things like memory management and out-of-order execution as out-of-scope. Some of them may even complain that Rust — a language that is very explicit about memory — is too implicit about other things.
package options
const (
OK = iota
WARNING
FATAL
)
import (
"package/options"
)
func main() {
ok := options.OK // iota value 1
useOption(ok) // OK
useOption(1) // ALSO OK
}
type errorLevel int
const (
OK errorLevel = iota
WARNING
FATAL
)
...I can do:
myerrorlevel := errorlevel(3458967478)
It would be good if there was some way of specifying that this type can only accept these values.
Thinking about it, it's the same as nil-safe types - being able to specify to the compiler that this type cannot accept a nil value.
From an external package you can only access the explicit values and the functions that accept them as parameters. The type is opaque for the developer.
enum Language {
Arabic `json:"ar"`
English `json:"en"`
French `json:"fr"`
Spanish `json:"es"`
// ...
} `json:"language"`
With generics Go is opening up the possibility of honest sum types. But, to be honest, I'm one of those people who are not upset by Go's error verbosity. The error shortcuts in Rust confuse me more often than not.
func foo() error { ... }
func bar() (string, error) { ... }
func baz() (string, int, error) { ... }
func doStuff() error {
foo()?
s1 := bar()?
s2, i2 := baz()?
}Anyways, you're entirely free to use a named return value in the second (or later) part of a multi valued := statement. An example is noted below, the clunky and obvious branches are there, but the intent and flow is obvious at a glance.
[1] https://go.googlesource.com/proposal/+/master/design/go2draf... [2] https://github.com/golang/go/issues/32437#issuecomment-51203...
Effective Go, on if: https://golang.org/doc/effective_go#if
It only promotes the concept that of using guard-clause style ifs that return an error and have no else clause, gradually eliminating each possible error condition. This style of programming was later termed in the Go community "Aligning the happy path to the left"[1][2]
[1] https://medium.com/@matryer/line-of-sight-in-code-186dd7cdea... [2] https://maelvls.dev/go-happy-line-of-sight/
I wholeheartedly agree that this style makes code more readable and I have adopted it - and encouraged others to adopt it - in other languages besides Go. This style would stil be followed with the try or check/handle proposals. For instance, let's take the example:
f, err := os.Open(name)
if err != nil {
return err
}
d, err := f.Stat()
if err != nil {
f.Close()
return err
}
codeUsing(f, d)
f.Close()
return nil
f := try(os.Open(name))
defer f.Close()
d := try(f.Stat())
codeUsing(f, d)
f := os.Open(name)?
defer f.Close()
d := f.Stat()?
codeUsing(f, d)
Not sure how it’s an issue, you just could not use the shortcut for this “not common” situation, or the case where the result is an actual product.
That Rust added ? Did not make every developer forget how to process values.
> The error shortcuts in Rust confuse me more often than not.
I have a hard time understanding that. The desugaring is hardly complicated, and the semantics are straightforward.
But the whole point of Go is that code is more often read than written, so legibility is paramount. Having a `?` operator alongside the traditional `if err != nil` would be very confusing.
Are you confused by unnamed returns? By the ability to declare variables initialised or not? By there being two syntaxes to declare variables? By it being possible to elide types in declarations or parameters lists?
Go is already full of shortcuts which I would say are more confusing and less useful than`?`
```x := potentiallyBuggyFunction ? genericErrorHandler("log message here", http.StatusNotFound,...)```
Where the `?` is kind of like a null coalescing operator, it might help?
One thing that I miss was an immutability story. You have values/references and, again I’m a beginner, I’ve the impression that passing references are favored. Wouldn’t it be nice if the compiler was able to check if you can mutate a reference or not? There are some proposals [1] that are discussed [2], that can be a nice evolution.
[1] https://github.com/romshark/Go-1-2-Proposal---Immutability
Initially I was taken back by the language, I felt we'd stepped back 15 years in language and design principles. I decried the lack of many of these features.
Today I feel more like the language has simply been misunderstood, and possibly because of its association with Google, wrongfully applied in places where it isn't the right fit.
It seems to been adopted, in many place as a fast system level alternative to higher level languages such as Kotlin or C#, when in actuality I think its niche is in providing a safer, more convenient alternative to C.
Great for smaller, performance critcal applications, or things that where the bundle size is important, such as operating system utilities, parts of the networking stack, small fast lambda functions, caching servers, etc. But not domain heavy application servers.
So with that in mind im not sure how I feel with GO current roadmap. Its now a language being adapted to be more suitable for applications I _feel_ it was never intended for.
I can work with Go when I have to. But it's not my main language. I appreciate the low barrier to entry. There's a ruthless pragmatism to the language that just means that whether you like it or not, that's how you do things in Go. Go-fmt is a great idea. Shuts down discussions about style that are fundamentally a waste of everybody's time.
The suggestions that the author makes are unremarkably reasonable and at the same time controversial to people used to the Go way. People get entrenched that way. It's like tabs vs spaces or curly braces vs. begin/end (or parentheses).
Enums are a good idea though. Should be entirely controversial. But clearly people have dismissed that idea because it is such an obvious and easy thing to do. I'd be curious about the reasons for that.
Error handling has been controversial since the language emerged with people loving it and hating it. IMHO it would clean up the language a lot if they acknowledged the simple reality that the most copied bit of go code is basically people deciding to not do anything substantial with the error other than logging it. Having the option to do something productive is nice. Making it required to spell out that you are not going to is silly. Java had a similar thing with checked exceptions. That's thankfully missing from Kotlin (which I use a lot) and it streamlines code quite a bit without really affecting stability. You still deal with the error somewhere. Just not right here right now.
I mention Kotlin here because I'm eagerly awaiting Kotlin native getting to the point where it becomes stable enough so people can write command line tools in it like people do with Go. JVM based Kotlin or Kotlin script are just not great for that (startup overhead, dependencies). Kotlin native actually works well on IOS currently but it is a bit lacking for command line usage; mainly because of lacking functionality in the library ecosystem. Just not a huge priority for Jetbrains apparently. But very fixable.
E.G. any case (including default) in a switch statement, or anywhere else, could have a keyword such as FORBIDDEN and at compile-time a check would result in: A warning if it is unprovable this is unreachable with a fatal exception if it is reached during runtime; an error if it is reachable (in the default case for your input handling this would inform you of the need to update the code); nothing (it is proven OK).
If there are possible values you don’t want to handle that’s totally fine, you just need to be explicit about it by having a blank default handler or similar.
1. The "default value" approach as Go, or how Java treats primitive types: It's simpler and safer.
2. Universal nil, or how Java handles non-primitive types: Bad idea. It's Sir Tony Hoare's "million-dollar mistake". It leads to unsafe code and feels pretty backward for a modern language.
3. Union/sum type/algebraic data types: The Haskell and Rust approach. It solves the problem faithfully but does introduce mental overhead. Whether it's suitable for languages like Go which strive for simplicity is subject to debatable.
As for Enum and Exception, in principle, it's the same situation that there are three camps of ideas:
1. A minimal approach which is the Golang we have. For enum is there is no enum. For error handling is treating errors as return values.
2. A mainstream approach like Java and alike. For enum is enum like in Java. For error handling its exceptions and ad-hoc syntaxes like "?." operators.
3. A full solution like Rust and Haskell. For enum, it's ADT. For error handling it's also ADT-based Either type and monadic operators.
It's interesting that how often these three camps talk past each other, and even more challenging to hold the community together and make the right decision for the language.
That is literally how enums work in most languages AFAIK. Even in .NET an enum is just an integer and you can in fact assign any integer value to an enum and therefore break things.
Proof: https://dotnetfiddle.net/lp51Dh
Re error handling:
In my opinion people who complain about having to write too many if statements in Go because of error handling just don't get it. I rarely ever want to bubble an err all the way up in my stack. In most cases I can either handle my error directly or if it's an error which cannot be dealt with at the current level then often I actually want to wrap it into a more high level application error and log the low level error or map it to something meaningful, distinguishing internal errors from user errors. People who look for a simple syntax to not deal with errors and just let them bubble all the way up simply do it wrong and need to learn how to write good code.
return SomeStructName{}, err
Concise error handling would be "nice to have" but to be honest, I prefer Go's current error handling over exceptions, most of the time.
I gather that Go people freak out over syntactic sugar like `cout << "foo\n'` but they've thrown away the ability to better express mathematical notation.
And the language itself cheats and internally can do that and has the kind of almost useless complex type, but you can't extend it to add vectors, quaternions and octonians that use algebraic operators.
You don't need this for writing a grpc web service, but if you start doing ML or game programming this becomes important, and I think there's one of those koan quotes out there about how the code you write should strive for clarity, and I'd argue that function calls are less clear, and the reason why we write `1 + 2` instead of `1.AddInt(2)`.
Point 2. It's not clear what the operator '?' does for a person new to the language. It will introduce confusion.
Point 3. Yes, I don't even know why it has to be that way.
I mean, it's not clear what pretty much anything (except the most basic stuff) does for a person new to the language. Has to be documented, like anything else.
https://www.infoq.com/presentations/Null-References-The-Bill...
I get that a lot of this difficulty stems from Go being a compiled language, but I still wish that writing tests were easier. (I'm really thankful that Ginkgo and Gomega exist though!)
There's no such thing as a struct name. Structs are unnamed. The only way to have a zero valued struct... is, obviously, to instantiate it with its members as zero values. Exactly what struct{members}{} is.
`type X struct { age int }` just makes X refer to a particular type of a struct, but this also works:
var john = struct { age int } { 16 }
The struct type very literally has a name:
type X struct { age int }
print(reflect.TypeOf(X{}).Name())
I encourage you to re-read the language spec, paying particular attention to the relationship between "defined types" and "underlying types".
1. A struct is a sequence of named elements, called fields, each of which has a name and a type.
// An empty struct.
struct {}
// A struct with 6 fields.
struct {
x, y int
u float32
_ float32 // padding
A *[]int
F func()
}
The new type is called a defined type. It is different from any other type, including the type it is created from.
type TreeNode struct {
left, right *TreeNode
value *Comparable
}
type X struct {} is simply a type named X, of underlying type struct{} which (I'm refering to the struct here) cannot have a name - because it is an ordered sequence [(name, type), (name2, type2), ...], not a product type (X, [...]).
I don't know why you bring methods to this, as if methods are somehow connected to structs. In your reflection example: Name() returns:
(1) the name of a defined type (2) the empty string for all other cases
It is designed for all types, and is of no particular meaning to the struct type, more than it is to int64 or float64. Would you say "ints have names"? You can create a type with a name and/or methods having such underlying types too.
I don't wanna nitpick in any case - my point was structs are not an exception like OP mentions. The brace syntax is a composite literal, and maps, arrays, slices all use it. The latter just don't use it for creating zero valued objects, because they are dynamic and are nil valued.
Both declaring a value without a specific value (var john struct{} or var john X) or a literal (struct{}{} and X{}), seem concise to me. Unless we bring inference to the table:
map[string]struct{age int}{"A": {15}, "B": {18}}yea, there's a lot of error handling code in Go, but a lot of the time its not just "if err != nil return err". errors are a simple interface type with an 'Error() string' function. This is NOT ENOUGH METADATA for real usable errors a lot of the time. Lots of codebases are littered with stuff like 'if err != nil return errors.WithStack(err)' simply to attach a stack-trace to an error coming from an external dependency. Sure, the enhancement proposed in the article doesn't preclude you from continuing to do that, but it doesn't solve the actual shortcomings of errors either in my opinion.
Errors in golang != exceptions common in other languages, I get that, but that doesn't mean I want to have to remember to attach standard metadata to every possible place an error bubbles up in my application. It's a consistency and simplicity problem too since some places you can just bubble an error up transparently, and sometimes you've got to wrap it with metadata for reasons that aren't obvious just by looking at the immediate error handling code. Thats a problem.
It's the same way C does it, other than saying "const" instead of "enum". Explicitly casting int -> enum is "recommended" but not required or enforced.
I think the problems with Go becomes obvious when you spend some time reading Go code. Try reading the code to Kubernetes and so on.
Switch operator a<b ? "neg" : "pos"
A coalesce operator myval ?? "default value"
generic map/reduce/filter
I'm happy generics are coming to Go, but living without them hasn't been difficult. (Like the article author, I feel like I'm jumping through more ugly hoops due to lack of a proper enum type than I am due to lack of generics.)
Programming languages are a human phenomenon, as is their success. They are popular in spite of their flaws, not because of them.
As a web engineer you should look at rescript/reason before making claims about compile times.
Also many system-ish programs, CLIs and the like also don't really need complex structures or benefit from authoring them + the algorithms that traverse them in a program-specific way.
That said, I'm not a fan of Go but generics are far from the only reason why I'm not a fan and it could probably exist as a somewhat useful language without them indefinitely.
There are times when you wish you had them, but they are typically conversion type functions. Those just don't happen that often for a lot of applications.
It's easy to show some contrived add func, and how you need one of every single number type in Go. But in practice, people rarely ever need such a function.
If you look at Java and C#, neither had generics either. You used casts, just like Go. It's safe, because the casts are safe (unlike void pointers). People started using Java and C# anyway, even though they didn't have generics.
Java then added generics in J2SE 5.0, which came out in 2004, when Java was nearly 10 years old. A year later, in 2005, C# added generics as well, with the release of C# 2.0.
Given that Go is about 11 years old now, it doesn't seem so out of place.
That is one of the reasons why the CLR already had most of the infrastrucure for proper handling of generic code instead of being a compiler only sugar.
"How generics were added to .NET"
https://mattwarren.org/2018/03/02/How-generics-were-added-to...
The way nil works is pretty simple. Unexpected cases are cases that didn't understand how nil works.
Errors as values is how the language works, and how I prefer it. No magic, make sure each step works. When I hammer a nail, I can tell immediately if I miss it, and deal with it. I don't have a catchall that sees I wasn't wearing safety glasses and makes some random hammer error.
Then again, I also don't think generics are a good thing, so maybe I'm just against change to the language at all.
I mean sure, but wouldn't it just be strictly better if there was just not null values and you specifically had to handle the present vs non-present case (i.e. Haskell's "maybe" type or other sum type construct)? Any non-trivial codebase is going to have programming errors where "nil" crashes the program where the compiler could literally have told you that your code was incomplete.
> Errors as values is how the language works, and how I prefer it. No magic, make sure each step works. When I hammer a nail, I can tell immediately if I miss it, and deal with it. I don't have a catchall that sees I wasn't wearing safety glasses and makes some random hammer error
Not following the analogy, but seems to rely on writing perfect code the first time, which is not possible. Nor are any of the proposed improvements "magic".
> Then again, I also don't think generics are a good thing, so maybe I'm just against change to the language at all.
With respect, it definitely sounds like it. Which is fine, but there's serious design problems with go that can be improved over time, and it's not a personal affront.
Option, for example, can be one of two types. Something or Nothing. And if the Option is Something, then the value is embedded in the "Something" variant of the type.
The callee has to match on the variant to extract the value. This provides more safety than nil does because you must specifically extract the value from the "Something" variant (and that extracted value can never be nil). You cannot extract the value from the "Nothing" variant (compiler error). So therefore, calling a function that takes an Option, and specifying Nothing is equivalent to nil (expresses intent to provide no value) with greater safety.
It just changes the onus somewhat.
A well written Go program will check for and handle nil up front. That an option type is 'safer' only helps enforce good habits.
I'm not against the idea, I just don't see the big benefit to a programmer who understands what he or she is working with. Unless the value is solely not having to think about nil cases, in which case I'd argue the value is very little.
Non-nullable types and compiler enforced checking of present/non-present values leads to more readable and more maintainable code. And there’s a mountain of CVEs, and bugs in go programs, to prove it. Maybe you just have to experience it.
Honestly a language designed in this century with nil values is tragic compared to what exists in other languages.
It enforces compilation result not habits. "to a programmer who understands what he or she is working with" - one day you'll learn that everyone makes mistakes regardless of experience, but it's better to accept and adjust to that fact before it bites you ;)
If you don’t know about that, please educate yourself, or at least ask questions, instead of replying with nonsense.
Same with “errors as values”, nobody is arguing about or against “errors as values” (then again I can’t say I know any language where that’s not the case, possibly VB? Exceptions certainly are errors and first-class values so they’re errors as values).