Handling exceptions in Python like a pro | Hacker News Reader

Handling exceptions in Python like a pro

blog.guilatrova.dev

46 pointslatrova25 comments

    try:
        ...
    except Exception as e:

Don't catch Exception at a low level. Catch OSError, which now includes EnvironmentError and IOError. This should cover all the things that can occur for external reasons. If you get some unexpected error such as a SyntaxError or TypeError, for example, that indicates a bug in the program. At the outermost level, catch Exception, and log a bug with a backtrace if an unhandled exception reaches that level.

An exception is not an error code.

Some good practices, reminds of Effective Java.

Couple nuggets as well:

  * just `raise` implies re-raising the current exception
  * `raise MyException() from ex` continues the stack trace
  * `logger.info(..., exec_info=True)` can add a stack trace to non-error level logs

Exception chaining (`raise from`) was new to me but after reading the docs it seems to be the default behavior in except/finally blocks, so the examples from the article don’t actually add anything. It doesn’t hurt to be explicit though.

https://docs.python.org/3/tutorial/errors.html#exception-cha...

Using "raise foo from bar" changes the message between the exceptions from "During handling of the above exception, another exception occurred:" to "The above exception was the direct cause of the following exception:"

So yes, these days the whole "raise from" construct is nearly pointless. Used to be that exception chaining wasn't automatic though.

I believe there’s a subtle difference in the meanings. You can raise or raise from. In either case the chaining means you get to see both messages. But one implies the original error is the cause. It’s so subtle that I don’t recall which is which!

When using a tool like Sentry, you might want to logger.exception(e) instead of a string error message, that way the whole stack trace with helpful debugging information (local variables...) is included in Sentry.

You might also want to use formatted strings too as it allows some logging tools to group the messages together.

logger.exception is basically a shortcut for logger.error(exc_info=True), so there's no need for explicitly assigning e as an error message.

And if you look carefully you’ll find that doing logger.exception(e) will just cast e as a string, which is just the message in the e. So you end up just sending the message twice (along with the stack trace).

Instead you can do logger.exception(“something helpful to give a little more context at a glance”)

I disagree with most of this advice. I think Google's Python Style Guide provides exception guidance that leads to clearer code.[0]

I agree with Animats's comment[1] about not catching the base Exception class, and the Google guidelines explicitly prohibit this, except at the outermost scope. One obvious flaw in catching Exception is if that if the user hits Ctrl+C to kill the application, but the code happens to be executing the ReceiptService at the time, the app will confusingly raise a ReceiptGenerationFailed exception instead of KeyboardInterrupt. (Edit: this isn't true, thanks to child comment for correction)

The final code snippet shows four function calls within the same try even though the functions throw a disjoint set of exceptions. It's unclear to a reader at the callsite what exception handlers corrrelate with which functions.

  except OrderAlreadyInProgress as e:
    logger.info("Aborting emission request because it is already in progress!")
    return {"order_id": order_id, "order_status": e.order_status.value}

To me, it feels sinful to use an exception to effectively "return" a value that the caller needs. I only use member fields in the exception to describe what went wrong. The original "bad" version of the code returned this information to the caller without raising an exception, and that felt more intuitive to me.

  else:
    return {"order_id": order_id, "order_status": order_status.value}

I don't understand the purpose of the else at the end. I would just write the return within the try as:

  return {"order_id": order_id, "order_status": status_service.get_order_status(order_id) }

[0] https://google.github.io/styleguide/pyguide.html

[1] https://news.ycombinator.com/item?id=27217301

https://docs.python.org/3/library/exceptions.html#KeyboardIn...

`KeyboardInterrupt` inherits from `BaseException` instead of `Exception` exactly in order to avoid this situation.

Oh, thanks for the correction. I remember running into this myself, but I must have done:

  except:

One more thing: avoid broad catch clauses. Some exceptions ought to break things, like KeyboardInterrupt.

By the way, you probably shouldn't wrap code lines on mobile. The code was difficult to read on mobile.

Some copy paste from previous posts

Exception based error handling is so bad and unsafe that adopting functional error handling with Either, Try etc as implemented by functional addon libraries for many languages, while not yet common, in time it will become the new default even in OO languages. (just like it's been the default in functional languages for decades)

Functional error handling types are much simpler, safer and more powerful.

Simpler because they don't rely on dedicated syntax- they're just regular objects no different to any other object.

Safer because unlike exceptions, they force callers to handle all potential outcomes, but no more. (no risk of ignoring errors and no risk of catching a higher level of error than desired, ubiquitous bugs in exception based error handling)

Powerful because they support map, flatmap, applicative etc, making it easy to eg chain multiple computations together in desired ways, which is unwieldy and bug prone when using exceptions.

    What is wrong about dedicated syntax

It adds complexity to the language! It could be that, when learning Java, Kotlin and any other language, we learn that methods return objects... and that's that. No weird dedicated syntax and magic, special treatment for returning anything other than the happy path, and the HUGE complexity that comes with it, eg the dedicated syntax itself and how it behaves, differences between checked and unchecked exceptions, hierarchies of exceptions etc etc.

    that makes our lives easier and our work more efficient

But that's our point, it doesn't.

Exceptions based error handling is unnecessary, hugely complex, doesn't compose at all, obfuscates or straight up hides what can go wrong with any given call, so leads to countless trivially preventable bugs... I could go on. And after decades of use, there's still no consensus about what exceptions should be or how they should be used. Exceptions are a failed experiment and I have no doubt that in ten years, Java, Kotlin and many other languages will acknowledge as much and move away from it the same way Joda Time outcompeted and replaced the horrible Java date and time library.

I've had great success with `Either` in Typescript. For each layer in the application I created ADTs representing the possible errors and used them in `Either`s. Each layer maps the lower layer errors to it's own.

From what I’ve seen so far, in reality people tend to overuse the `try!` macros, which leads to the same old bubbling exceptions semantic, but more verbose, less performant, and lacking stack traces.

Right, with exceptions you can `catch throw` to always get a stack trace of when the exception is thrown

Except that error returns such as Either have their own problems. What is the main complaint about checked exceptions in Java? Yes, handle or _declare_ them in method signatures: adding a checked exception to a previously non-throwing method breaks all callers. What happens when a previously non-erroring function must be changed to be able to report some error? Yes, the same: change the signature to return Either and fix all callers.

And then what about conditions like division by zero, or arithmetic overflow? Are arithmetic _operators_ supposed to return Either/Option instead? Or just crash the program? Which may be a long-running server and only one request of many got corrupt?

While not very ergonomic, I still prefer exceptions way more than return values. Ergonomics could be fixed by tooling; hopefully that will improve if developers start to demand it.

Finally, even functional languages such as SML and Ocaml have exceptions where they are also even recommended for control flow such as breaking out of deep recursion.

> And then what about (...)

If seems you've missed the whole point of suggesting the use of Either/Result monads. It's entirely irrelevant if you have legacy code that does not support returning error codes a certain way. The whole point of Either/Result monads , specially regarding exceptions, is that they provide a clean way to handle errors (and not exceptions) in the normal code path by explicitly making them a part of a function's codomain, and without relying on magic bubbling up.

And no, Either/Result monads are only designed to handle expected results, not exceptional behavior. Perhaps your main source of confusion is conflating exceptions with an error condition. In that case, think of Either/Result monads as "I made a HTTP request and the response was a 200 or a 500 status, thus the code path follows accordingly" and exceptions as "I made a HTTP request and I got an OutOfMemoryException".

I just looked into this following someone else recommending Either and my knee jerk reaction is the same as yours.

The nice thing about exceptions is that you get to take a large block of code and treat it as a whole fail-able unit. In terms of a control flow structure, being able to raise deep and goto nearer the top of the stack removes large amounts of boilerplate code, which, in my experience, makes the code more readable.

I use exceptions sparingly and deliberately. And when used like that, I find them to be a helpful feature that improves the codebase.

Why cant you do what haskell does? Or you can still use exceptions alongside either and options. I think the problem with python is they get used for everything.

Two sides of the same coin. If error values like Either take over people will probably create abstractions (macros, monads, IDE plugins, whatever) to automatically propagate error values up the stack, then we're back where we started.

I'm dealing with some code now that loves to use error values like "not found". Error values can be poorly done too. I've never seen exceptions quite so poorly done, they always at least have a stacktrace as a start to figuring out what they mean. I don't know where to start when I get the string "not found" and nothing else. Guess I'll use a debugger? I'll probably find it was division by zero or something, then got turned into the string "not found".

Exceptions carry stacktraces which are really useful, and hard to replace with a string or any user defined object.

And "panics" will always exist, so the idea of errors going up the stack through an alternate channel will never go away, and there needs to be ways to recover from those panics. So really, the choice is we use exceptions (or "panics") AND error values, or just exceptions. Using only error values is not an option.

The reason so much low level code is written in C is that not having exceptions forces one to write robust code. Panics stop the kernel. Sigsegv kills the CPython process. Exceptions make it extremely difficult to have that level of robustness.

To be more explicit, C stays around despite the memory unsafeness because the code written without exceptions just works so well. In the 90s, I worked writing in C long lived servers that had the philosophy to never ever exit because of bytes read from the network and always trigger a core dump and exit on any internal application state inconsistency. In production the process would be restarted and the stack trace emailed to the author, with core available if needed. It was possible and pretty routine to have code that would run hundreds of days, years even once the feature was basically done, without restart. Exceptions make that degree of reliability really hard for us simple minded engineers to achieve.

Setting aside my personal opinions. I think it's safe to say that there are pros and cons to exceptions versus error values.

But, I think it's even more safe to say that error values are not a silver bullet that inevitably lead to robust code. As mentioned I'm personally dealing with some very poorly done error values right now. I resent the idea that error values "force" robust code (I wish that force was in effect on my own project), or "just works so well."

They're really not that different, the two approaches.

Reply on news.ycombinator.com