The Ultimate Guide to Error Handling in Python
blog.miguelgrinberg.com
blog.miguelgrinberg.com
The article says basically "LBYL is bad", but this isn't a good description of what the author does in the later examples. Following "EAFP" without exception is also really bad.
The simple policy is that you should use "LBYL" when you're dealing with local state that can't change out from under you, especially when it's just properties of your local variables, but you need to use "EAFP" for anything that deals with remote state. Function calls can go either way, depending.
Blanket advice against LBYL leads people to use try/except blocks to express basic conditionals. If you want to ask something about your local variables, just ask it in a conditional --- don't make the reader infer "ah, these are the situations which will raise the error, and therefore we'll go into this block if the values are this way". If you want to do something when a key is missing from a dictionary, don't write:
try:
value = table[key]
except KeyError:
return default_value
if key not in table:
return default_value
value = table[key]
The example the author gives about interacting with the file system is a good example of where indeed you really should use EAFP. The file system is remote state that your function does not own. Similarly if you're doing database operations, calling a remote API...lots of things.
There's various middle ground when you're calling functions. Often you don't want to worry about whether that function is going to be interacting with remote state, and you want to treat it totally as a black box, so you just use EAFP. But if the function documents really clear pre-conditions of your variables you can just go ahead and check, it's better to do that.
Brett Cannon, one of the Python core devs, wrote a blog post using exactly this dict KeyError example in 2016 [1]. It concludes:
"The key takeaway you should have for the post is that EAFP is a legitimate coding style in Python and you should feel free to use it when it makes sense."
[1] https://devblogs.microsoft.com/python/idiomatic-python-eafp-...
If it was something like except KeyDoesNotExist: or KeyNotFound, it would make more sense for me, because it seems hacky to consider it an error where it's a normal default to some value behaviour.
Using try/except for conditional logic gives the developer a spurious choice between two different syntaxes to express the same thing. The reader is then asked to read try/except blocks as meaning two different things: either ordinary expected branching in the function, or handling exceptions.
I think it's definitely better if we just use conditionals to express conditionals, and try/except for errors, like every other language does. Here's some examples of where this duplication of syntax causes problems.
* Exceptions are often not designed to match the interface well enough to make this convenient. For instance, 'x in y' works for both mapping types and lists, but only mapping types will raise a `KeyError`. If your function is expected to take any iterable, the correct catching code will be `except (KeyError, IndexError)`. There's all sorts of these opportunities to be wrong. When people write exceptions, they want to make them specific, and they're not necessarily thinking about them as an interface to conveniently check preconditions.
* Exceptions are not a type-checked part of the interface. If you catch `(KeyError, IndexError)` for a variable that's just a dictionary, no type checker (or even linter?) is going to tell you that the `IndexError` is impossible, and you only need to catch `KeyError`. Similarly, if you catch the wrong error, or your class raises an error that doesn't inherit from the class that your calling code expects it to, you won't get any type errors or other linting. It's totally on you to maintain this.
* Exceptions are often poorly documented, and change more frequently than other parts of the interface. A third-party library won't necessarily consider it a breaking change to raise an error on a new condition with an existing error type, but if you're conditioning on that error in a try/except, this could be a breaking change for you.
* The base exceptions are very general, and catching them in code that should be a conditional runs a significant risk of catching an error by mistake. Consider code like this:
try:
value = my_dict[some_function()]
except KeyError:
...
Because you must never ever ever call a function inside your conditional try block, you're using a syntax that doesn't compose properly with the rest of the language. So you can either rewrite it to something like this:
value = some_function()
try:
return my_dict[value]
except KeyError:
...
The fundamental thing here is that 'try/except' is a "come from": whether you enter the 'except' block depends on which situations the function (or, gulp, functions) you're calling raise that error. The decision isn't local to the code you're looking at. In contrast, if you write a conditional, you have some local value and you're going to branch based on its truthiness or some property of it. We should only be using the 'try/except' mechanism when we _need_ its vagueness --- when we need to say "I don't know or can't check exactly what could lead to this". If we have a choice to tighten the control flow of the program we should.
And what do you buy for all this spurious decision making and the very high risk of very serious bugs anyway? Why should Python do this differently from every other language? I don't see any benefits in that article linked, and I've never seen any in other discussions of this topic.
if some_function() not in table:
return default_value
value = table[some_function()]Let's say you're looking at some code like this:
if value in table:
...
If you instead have:
try:
return table[value]
except KeyError:
...
if key not in table:
return default_value
value = table[key]
value = table.get(key, default_value)
...
return value
In case you disagree, I'd be happy to hear your thoughts!
I'm not sure what you mean by this? Python has the "raise" statement [0], which "throws" an error from within a function or other code.
[0] https://docs.python.org/3/reference/simple_stmts.html#raise
Python does not have that.
Plus, the typing annotations could presumably be expanded to include some notification to declare raised exception types explicitly.
The ironic thing is that Checked Exceptions aren't particularly well-loved in Java. For example, Kotlin (which is like Java++) did away with them.
I think what I've come to understand is that you can treat errors you don't know about as non-recoverable, because that's most likely what they are anyway if they aren't readily documented. Let them bubble up basically, and handle them at the top level if necessary to prevent crashes in prod/make sure they are logged correctly and so on.
For this Rust style error handling can be interesting, eg using poltergeist (https://github.com/alexandermalyga/poltergeist) or result (https://pypi.org/project/result/).
With normal exceptions, I have to use my meat brain to figure out if I've handled all the exceptions that might come up, whereas mypy can just tell me if I've handled the result object correctly
That said, I don't use either library since, in my experience, you need them to be used all the way down the stack to actually get the benefit.
The point of exceptions is exactly that you don't want to handle all that might come up -- you need to handle only those that you actually expect and know how to handle. Everything else is left to bubble up and either be handled by a custom, generic top-level handler, or by the language's default mechanism (in Python's case, dumping a stack trace).
print(f"Unexpected error: {error}")
sys.exit(1)
Is all this doing is making sure the exit code is 1 instead of some other non-zero value?
I'm not seeing how this is any better than just letting the program hard-crash.
(I am asking in earnest. I'm getting back into Python and wanting to know what's generally preferred)
It's often more user friendly to print a short human-readable message (or nothing) than to dump a huge stack trace. YMMV, if this is an internal dev tool then the opposite might be true!
I wrote a small utility library for exactly that kind of "railway oriented programming" if you're curious about what it'd look like in python: https://github.com/benrutter/ufo-tools
I tried it once in an sqlite DB connector with some business logic and simply checking stuff like
res: DBException | Result = db_handler.some_business_logic()
if isinstance(res, DBException):
return res # you can also log or even raise if this function isn't returning exceptions as values
# guaranteed to be Result type here
- https://gitlab.com/meehai/drpciv-flask/-/blob/main/be/db_han...
- https://gitlab.com/meehai/drpciv-flask/-/blob/main/be/app.py...