Exploring Error Handling Patterns in Go
8thlight.com
8thlight.com
Errors should be "decorated" (wrapped, contextualized...) in 99% of the cases.
In the end you get errors that describe step by step what your program tried to do and why it failed, for example:
* could not load profile: could not open file: permission denied.
* could not download profile image: could not open URL: HTTP GET failed: network is down.
This has many advantages:
1. Much more readable than stack traces (especially if they include source file and line information or exception class names: users don't care about those.)
2. Errors are still easy to grep in code to work out the program flow (the stack trace, basically.)
3. When reading the code, you can see from the error context strings what the code is actually doing. Basically it serves a function of comments and (unlike comments) error strings remain up to date.
It is definitely verbose, especially the not equal nil part, as it's a result of Go attempt not to have special cases. Also it's a pity that errors can be silently ignored: maybe Go2 could be stricter here.
Overall, I think this is one of the best approaches at error handling.
I'll overwhelmingly prefer an always-correct stacktrace over a hand-recreated one that sometimes collapses multiple branches into a single ambiguous on. At least then the devs can help me when it fails. And stack traces and concatenated strings are in no way appropriate error responses for humans unless you're expecting them to be able to navigate the source code, so neither does anything for the "provide a helpful error message for non-programmers" problem.
---
this is why stuff like https://github.com/pkg/errors exists. wrap at the deepest level / where the error originates, and it's relatively rare that you need to add context at higher levels. If you want user-friendly errors, you need something dramatically more sophisticated.
Returns go straight up the stack only if you choose them to. Caller doesn't have to propagate errors and return immediately, it can hold onto them and/or process them in the natural place they occur, they are just values. The returns from your function can be found with "grep return".
Exceptions, by default, break the natural control flow unless you wrap everything with try/catch. Even then, a lot of constructs won't be very natural and you really will have to get out of your way to identify the source of the exception, which very frequently is more important than it's type.
(Panics of course can cause similar thing in Go but that's the reason why they should rarely be recovered and not used as a value propagation mechanism.)
func thing(arg) {
otherThing(arg) // does this return an error you're ignoring?
}
Granted, you could turn this into a compiler error. But it's not currently.
---
Anyway. Given that so many not-prevented-by-the-typesystem operations panic, I don't think it's reasonable to assume that any func call will not panic, especially not in the future since they may change. So you already have to program with `defer` to maintain your invariants, which is exactly the same as with exceptions, except you now have to deal with both possibilities for nearly every func.
The built-in tool does not even warn about unused errors... not `go build`, and not `go vet`. What's more important, an ignored error or an unused import?
Anders give an interview in 2003 [1] where he talks about how C# looked to learn from Java's checked exceptions. His conclusion was basically that, in their evaluation, 9/10 exceptions cannot be handled beyond some generic top-level handler.
If this observation is correct, and it certainly aligns perfectly with my own, then bubbling makes a lot more sense.
Note that, with error return values, you can emulate bubbling (which is what most Go programmers end up doing). And with exceptions, you can emulate return values. The question is what's the most common default? And, again, according to Anders, as well as any project I've ever worked on, bubble-by-default is overwhelmingly the most useful thing to support cleanly.
The only way Go's approach makes sense is if you consider it's original goal (system programming) and MAYBE (i don't know, I'm not a system programmer) for such systems you can/need to handle each error. Except that's not really how Go is being used now, so...
The fact that I don't even have to think if the function call I'm looking at can throw and if I should catch it or not outweighs everything.
Edit: Also, there is no parallel control flow. Languages with exceptions have union-type return values, and every statement is implicitly followed by the equivalent of: if err!=nil return nil, err. The fact that in Go you have to type that makes Go cumbersome, not smart.
Usually if something panics you don't want to handle it. (Other than at the http handler level, where you can just throw an InternalServerError and log the panic)
First and foremost, you can usually assume libraries won't panic, though it would be nice to have a tool (grep) to check for explicit panics.
I think errors as values cause you to always think about this, which makes you handle errors in a more sensible way, instead of just bubbling up. Sure, 90% of situations you will bubble up, but in my opinion it's still worth it.
> Thanks to panics, even in Go.
Fortunately, for some reason Go developers don't use panic like exceptions and recover them at library boundaries.
Yes, exactly like that. In real world programs, every function might fail, even the simplest ones (stack overflow, out of memory, interrupts, etc). No information is gained by declaring a specific function might fail. Also it's almost certainly a lie to declare: this function will never fail. So if every function might fail, why not just produce a union type <Result|ErrorData> for every function return type. Also, lets automatically check for the error case after each nested invocation, and cleanly unwind the stack (returning ErrorData again) on failure. This makes 95% error handling code go away. The ErrorData type uses a special return keyword ("throw"), and in rare cases, errors need to actually be handled, so the union's ErrorData type is exposed to the user code with additional primitives (catch). On all the code in between, the ErrorData type is just hidden behind the scenes.
The thing many programmers don't seem to realize is that a program is a model of a design in the programmer's mind. If the model goes off the rails of the expected design/behavior in any way, that should be considered very bad ASAP... or as many languages treat it, "exceptional".
Go errors out on unused imports, but you can type "import _ foo.com/unused-import" to not error out.
Why doesn't 'errors.New("asdf")' error out and require you to instead write '_ = errors.New("asdf")' to ignore the result
I think the real answer is not that it's intentional design, but rather that the original compiler was not powerful enough to implement that feature easily... and once go hit 1., it was impossible for them to add new warnings or errors because there are no warnings and errors are backwards incompatible.
Sure, that means developers use third-party tools for warnings because the go compiler refuses to ever have warnings (that compromises the pure beauty of the language obviously), but at least that means it's only the users that have to deal with the complexity of using more tools, the compiler developers can ignore it.
One of the reason is that you don't always want to check the error. The most common one is fmt.Println.
I would not like to always write
_, _ = fmt.Println("Hello, playground")
This is why you want unchecked errors to implicitly bubble up. Which is what exceptions give you, or perhaps a syntax with implicit error return values rather than Go's by-convention approach.
And if it means a lot in this case, wrap the function and use it instead.
For me, the main difference between Go's way of handling language and the rest of mainstream languages is that it makes error handling unmagical.
It literally says – errors are just like any other return values. Let's say, if you have function `sqrt` and return a value, and then call this function – you probably is interested in this return value and should handle it somehow (or mute with `_`). Now, the same applies for errors - if function returns the error, you likely to think how to handle it – do something in place or propagate up the stack.
There is also a cultural moment to this. As we mostly learn by examples, and most Go code has proper error checks (not equal to "proper error handling" but nevertheless), it makes newcomers to do the same as well, even while disagreeing with Go's way. I've heard from many devs that Go was the reason that made them appreciate proper error handling.
And honestly, I feel this too, and I think the reason is that in Go it's too easy to "handle errors properly". I never had this feeling with languages with exceptions, where I had to read whole books (!) just to learn how to properly use them and be confident in the way how I handle errors. (this is just an example, not the spark to start return values vs exceptions battle, just in case)
The code will indeed still be easily understandable on a line-by-line basis, but larger units (functions, groups of functions, modules) then become harder to read and understand due to large amounts of noise. As you progress along to more advanced, larger codebases, with novice-style code you just pile boilerplate on top of boilerplate.
Larger Go codebases remind me of old versions of Java where they went as far as to embrace all the boilerplate and call it 'design patterns'. It's sad how Go designers recognized that Java code tends to be hard to read and generally bad, but apparently weren't really able to discern why that is. They decided to just blame it all on inheritance and exceptions.
(FTR I'm not advocating for inheritance or exceptions here, but believing that those are the root of Java's problems and simply omitting them will somehow magically make a language better is just naive & shortsighted.)
Yeah, that seems about perfectly correct. Have you looked at novice's code? Is it easy to understand?
It's not about saving the compiler work at all, it's about saving the hundreds of humans who have to read your code after you the work of understanding the abstractions you created.
https://www.sandimetz.com/blog/2016/1/20/the-wrong-abstracti...
I'm not saying Go is the best of all possible worlds (I would like to see generic functions like map too, or things like sum types for errors), just that there are good reasons for the decision to exclude some opportunities to build abstractions (for example I'm happy go eschews inheritance), and abstraction is not an unmitigated good. I've seen far more bad abstractions built than code duplicated when reading code in any language, so limiting abstractions is not always a bad thing.
Besides the notion of "The wrong abstraction", which is sometimes used as a proxy for "Abstractions I don't want to learn", we're discussing about language level abstractions here. The article you quote criticizes user level abstractions.
Language-level abstractions have a decent enough track record that we can assess them. Go even uses some of them: GC is, after all, an abstraction.
There would also be a lot to say about what the "little" in "A little duplication" means.
The reality is most go programs are (1) very difficult to understand because error handling swamps their logic and (2) end up reinventing exceptions anyways, albiet poorly and (3) inevitably end up leaking resources because go's "error handling strategy" doesn't ensure resource cleanup.
We can observe this and measure this quite clearly in non-trivial go codebases.
Eventually the go dictatorship will relent and provide exceptions. At that point all the people who praise the existing broken model will happily praise the new approach and denounce the existing brokenness.
In my years of experience with the language, this is, bluntly, untrue. Go, used properly, is slightly more verbose than most comparable code in Python or Perl. If someone is writing code that is shot through with boilerplate in Go, then I would say that they may be using "Oh, Go just needs lots of boilerplate" as an excuse.
The problem isn't that Go lacks abstraction mechanisms; the problem is that you need to learn how to use the ones that are there and not sit there pining for the ones that are not. I find this to be almost exactly like learning Haskell; you need to learn to use what is there, not sit there pining for what you don't have. Also like Haskell, there are some particular points that it all comes together at once and hurts you, but, then again, there's some places in Go where I've had big wins using the language features too. It does cut both ways. (I've done some fun things with interfaces, and the pervasive io.Reader/Writer support, while not necessarily a feature of the language, can make certain things amazingly easy to do while still retaining incredible flexibility.)
As one example I went through personally, while by the time I learned Go I had a lot of non-OO experience, so I wasn't as stuck on inheritance as someone who only did OO-languages for the last 10 years would be, I still had to adjust to using a generally-OO language (by my standard of the term) that did not support inheritance. It has now been literally plural years since I missed inheritance in Go. (In fact, quite the opposite; I miss easy composition in my other OO languages! Yes, Virginia, it is possible to miss features Go has when using other languages, despite what it may seem like if you only read the criticisms.) But my first couple of months were a bit rougher before I internalized how the composition works and affects the design of your code.
Complaining that Go code is all boilerplate is like someone who tried Haskell but complains that it's just an especially inconvenient imperative language and you end up doing everything in IO anyhow. Nope... you have not yet gotten past your "Writing X in Y" phase. That's fine; there's a ton of languages and platforms and libraries in the world. If you didn't get a short-term payoff from using it, go ahead and move on. But you haven't attained enough mastery to go around slagging on the language/platform/library yet.
(And, again, let me say that, yes, it is somewhat more verbose that Python or something. If you've shrunk your Go down to that level, you probably went too far and are doing something ill-advised. But I find that in practice, for most tasks, it is not that much more verbose. There are exceptions, like heavy duty GUI code or (IMHO) scientific code; the solution is not to use Go for those.)
I agree that in some way, this is actually a good reason to support Go's tedious way of handling errors.
Yet, it's akin to avoiding functions (because stackframes are magical), or "for" loops (because their condition block is magical), or threads (because: magic). There is only so much a non-toy programming language should compromise in order to accomodate beginners. People may draw different lines here, but Exceptions (in garbage collected languages) are so completely unmagical, that the line should definitely not be drawn here. In fact, they do exactly what return nil, err does, thousands of times, over and over. In Go you can enjoy writing that code yourself. Also, Go's language designers accepted defeat when they had to add panics. I bet that beginners now just make the mistake of ignoring them instead.
I liked when I started with Go (mostly due to concurrency primatives, but errors were nice too). In my day job, many of the errors have a need for custom handling and I get that for "free" in Go and I am never surprised by a program crash because I failed to read the docs on a function and what exceptions it may throw (or undocumented exceptions it may throw due to one of its dependencies). I can see right in the signature that I have an error to potentially handle.
Suppose you have a function that fetches a model from your database. It can return an error if the given user doesn't have permission to fetch this model, or it can return an error if your db connection barfs for some reason. The calling function needs to be able to differentiate between the two errors. Most of what I've read on the subject makes it seem like people prefer to only ever check if err != nil.
The two options I've seen in the wild are:
1. Create a constant for a given error, like:
var ErrFetchForbidden = errors.New("FETCH_FORBIDDEN")
if err == ErrFetchForbidden {
return 403
} else if err == ErrFetchNotFound {
return 404
} else {
return 500
}
type ErrFetchForbidden string
var err ErrFetchForbidden = "error retrieving the user object"
return err
switch v := err.(type) {
case ErrFetchForbidden:
return 403
case ErrFetchNotFound:
return 404
default:
return 500
}
Create a custom error type, for example DB Error:
type DBError struct {
Temporary bool
NetworkBased bool
Cause error
}
Otherwise, you can use 2# with a twist, instead of matching on a type, you can do:
switch {
case isErrFetchForbidden(err):
case isErrFetchNotFound(err):
}
IsBadRequest(err)
IsInternal(err)
IsTimeout(err)IsTemporary also takes an error and does something like:
if err, ok := errors.Cause(err).(*DBError) {
return err.temporary
} else {
return false
}Actually, we unknowingly ignore returned errors much more often than we think, like when we call a function and opt out of assigning any of the return values to variables. Consider this function, which returns a single value (being an error).
func Failure() error {...}
Failure()
writer.Write([]byte("Hello")) // returns (n int, err error)
https://dave.cheney.net/2016/04/27/dont-just-check-errors-ha...
if err != nil return err
if err != nil return err
https://github.com/docker/cli/search?q=%22if+err+%21%3D+nil%...
https://github.com/kubernetes/kubernetes/search?q=%22if+err+...
https://github.com/coreos/etcd/search?q=%22return+err%22&uns...
https://github.com/influxdata/influxdb/search?q=%22if+err+%2...
The reality of Go's error handling is that you just implement exactly what exception bubbling does painfully by hand.
One important benefit of Go's error handling pattern is readability. With exceptions, it's not easy to see who handles it and where. There is indeed less code, and that's nice for the writer, but from the reader perspective, error handling becomes obscure. And from the quality control point if view, this becomes unsafe.
The philosophy is different when, for example the author of Ruby wanted to make coding fun for programmers and does a good job at it and Go is sticking to 'this must be right' approach and breaks some people's heart.
Personally I'd appreciate being more 'fun'.
The fun part is getting code written, and shipped. And it stays fun when it’s maintainable and production ready.
I’m having a lot of fun shipping Go code. :) I definitely can understand a codebase a lot faster than a random ruby one. That may be a personal thing but it works for me.
This is because to make it easier for the novice, there are all kinds of constructs that try to make the code imitate normal English. But coding software is a completely different thing than writing text, thus the English-like front is in fact a smoke screen that hides the real gears.
A small example would be the unless keyword. It completely throws me of each time I come across it, because it reverses the order of evaluation:
Do something, unless condition applies.
I read that from left to right, so in my mind "Do something" has already executed, but then I have to go back, because the condition might not apply. This get really 'fun' if the condition is something negative.
I like Go just because it way more simple. Even it is a bit more verbose in the error handling, everywhere else it is very minimal and clear.
Every function has a hidden "err" return value
Every function has an "exceptionExit:" block, by default it does just "return err;"
After every function call, an automatic "if err != nil {goto exceptionExit}" is added.
You can add an "exception" block to a function, it replaces the default.
Now you have function level exceptions in Go just by syntax sugar, without stack unwinding and without requiring new compiler functionality, just syntax sugar.
> One important benefit of Go's error handling pattern is readability
I beg to differ, Go's approach is similar to checked exceptions, Java's original sin. And just like checked exceptions, forcing the invoker of a function to handle the error directly is the wrong approach in the vast majority of cases. It just produces code noise and catch/wrap/throw style code, commonly found in old Java enterprise projects. This obsfucates the default path and makes middleware very hard to write.
> With exceptions, it's not easy to see who handles it and where.
Making errors part of the function signature encourages developers to handle them directly at the call site. Which is where most buggy and unreliable error handling is found. The default approach of safely unwinding the stack until you reach the http handler (or equivalent), returning 500 applies to error codes as well. It should be simple to do, automatic even, so novice programmers write robust code out of the box. Hence exceptions.
When you read code with a call to a function returning an error, you see how the error is handled. With exception, unless there is a try/catch close surrounding the call, you don't know where and how an exception is handled.
To me, throwing exceptions is like littering the streets. That feels fine for the one who does it, because he assume someone else will take care of the mess. But the problem is taking care of it, who, how when ? With big projects, this strategy is unmanageable.
Correctly handled exceptions don't make middle-ware easier to write or more readable. On the contrary.
You know that programs are not only http handlers, right ?
In practice you only see that errors get returned immediately. Most functions rightly give up rather than trying to handle errors because they don't know exactly how or where they're being (re)used.
A checked exception _requires_ an exception type specification. If you don't handle the exception locally, that is. It's this quality I refer to, when I say checked exceptions are similar to error codes: an API designer, without knowing the full context, requires the call site to do something about it, even if 9/10 call sites could not care less, _especially_ in big projects. Some other commenter linked to this interview [0], which elaborates on that problem.
> With big projects, this strategy is unmanageable.
Clearly, it's possible to have mantainable projects both with and without exception handling.
> You know that programs are not only http handlers, right ?
I find this somewhat condescending, but yes, I do know that. Most programs have system boundaries though, and might recover from quite severe error conditions there.
That's my point. returning the err until some caller above you handles it is unwinding the stack. You're just forced to do it manually at every single level of the stack.
Maybe; personally I find it increased clutter that obscures readability (much as do checked exceptions.)
> With exceptions, it's not easy to see who handles it and where.
Who handles it and where is the one thing that is explicit and readily apparent with unchecked exceptions. What can be harder to see with unchecked exceptions than with error returns or checked exceptions is who (other than the original source) throws it and requires consideration of handling it or ignoring/rethrowing it in the caller.
Errors are not just values, however, they're something much more specific: An uncaught error is a specific circumstance where the programmer's mental model was insufficient to account for all the state possibilities. Literally the introduction of the unexpected. And this should be treated as a very bad (or at least a very special) thing, as soon as possible. Hence, runtime exceptions. (Hence, disclaimer alert, I'm not a fan of Go.)
I’m not sure if I’d use it again today, but it was a fun exercise.