Hacker News Reader

Comment by amanj41 | Hacker News Reader

I think a good usecase for recover is in gRPC services for example. One wouldn't want to kill the entire service if some path gets hit leading to a panic while handling one request.

View on HN

Corporate gRPC services are written with "if err != nil" for every operation at every layer between the API handler and the db/dependencies, with table-driven tests mocking each one for those sweet sweet coverage points.

I would love a community norm that errors which fail the request can just be panics. Unfortunately that's not Go as she is written.

One thing that `if err != nil { return err }` lets you do, which panic/recover doesn't, is annotate errors with context. If you're throwing from 5 layers deep in the call stack, and two of those layers are loops that invoke the lower layers for each element of a list, you probably really want to know which element it was that failed. At that point, you have two options:

1. Pass a context trace into every function, so that it can panic with richer meaning. That's a right pain very quickly.

2. Return errors, propagating them up the stack with more context:

  for i, x := range listOfThings {
    y, err := processThing(x)
    if err != nil {
      return fmt.Errorf("thing %d (%s) failed: %w", i, x, err)
    }
  }

You can add arbitrary information by catching and rethrowing exceptions (which go panics, basically, are).

Go panic isn't really usable as catch/rethrow because it can only be done at function scope. To make them useful for that pattern, you need a scoped `try { }` block where you can tell what part failed and continue from there. Either that, or you need lots and lots of tiny functions to form scopes around them.

You don't need "lots and lots" of tiny functions, most of the time it's totally fine to just let the exception propagate as is. When you do need to add information, you will have to use a function, yes, it's an unfortunate feature of golang. Same with defers, the only way to scope them is to wrap them in a function, it's stupid, but this is golang for you

I mean yes, but I don't really like hand writing exception tracebacks via error wrapping.

That said... I did like a clever bit I did where you can use a sentinel error to filter entire segments of the wrapped errors on prod builds. A Dev build gives full error stacks.

That’s mostly just a stacktrace. You can add other information that wouldn’t be in a stacktrace, but you shouldn’t do it by string concatenation because then every instance of the error is unique to log aggregators. Instead, you need to return a type implementing the error interface. Which is not all that different from throwing a subclass of exception.

Yes that is common. I was more talking about the case where someone perhaps introduces a bug causing a nil pointer dereference on some requests, so the panic is not explicitly called in code. In which case you would definitely want the recover in place.

Instead of recovering from what clearly is a bug, why not fix that bug instead?

Some are of the opinion that that should be handled a layer up, such as a container restart, because the program could be left in a broken state which can only be fixed by resetting the entire state.

Given that you can’t recover from panics on other goroutines, and Go makes it extremely easy to spawn over goroutines, often times it’s not even an opinion, you have to handle it a layer up. There’s no catchall for panics.

This is a major pain in the ass. I was trying to solve the problem of how do you emit a metric when a golang service panics, the issue is that there is no way to recover panics from all goroutines so the only way to do that reliably is to write a wrapper around the ‘go’ statement which recovers panics and reports them to the metrics system. You then have to change every single ‘go’ call in all of your code to use this wrapper.

What I really want is either a way to recover panics from any goroutine, or be able to install a hook in the runtime which is executed when an unhandled panics occurs.

You can kind of fudge this by having the orchestration layer look at the exit code of the golang process and see if it was exit code 2 which is usually a panic, but I noticed that sometimes the panic stack trace doesn’t make it to the processes log files, most likely due to some weird buffering in stdout/stderr which causes you to lose the trace forever.

Reply on news.ycombinator.com