Exceptions for control flow considered perfectly acceptable
drmaciver.com
drmaciver.com
> Argument by meaningless blither. What distinguishes
> “exceptional conditions” from “control flow”? I have
> reached the end of a million element list. This
> happens one time in a million! That sounds pretty
> exceptional to me!
An example of misusing exceptions for control flow in this context would be throwing a "caching exception" to indicate that a cached version of the requested resource has been found and is to be returned to the client without any further processing.
Maybe exceptions were invented for handling errors, but now that we have them, what's a good reason not to use them for other purposes?
Dependencies between components should ideally be as explicit and as few as possible. Dependency injection is one good way of achieving this, but even global variables or singletons are more clearly self-documenting than exceptions. A piece of code that returns a value or makes a call to some functionality in another piece of code is easy to understand. If instead, that code throws an exception, then it sends the reader on a voyage of discovery throughout the codebase to investigate all the places that catch this type of exception, and which of those pieces of code can ever potentially be higher in the call stack than the piece that throws it.
The only reason we even put up with such complexity in the case of handling errors is that it is the one problem that is ubiquitous to every piece of software ever written and therefore merits its own first-class language support.
When you define an API in a language that has exceptions you should be careful to define the behaviour around how exceptions are used.
This is similar to how, when you define an API in a language that has threads, you should be careful to define your reentrancy and thread-safety guarantees.
http://www.ckwop.me.uk/Why-Exceptions-Suck.html
Not saying that this is the last word on exceptions, but these are the arguments.
Are you talking about a language that you (h2s) wrote? Or some other language like Java, Python, C#, Ruby, CLU or PL/I? Because I doubt that you KNOW the reason that exceptions were made a part of the language. And it doesn't matter anyway.
Imagine someone saying "the print statement exists so programs can output simple debugging statements to the console". It is mostly true, in the same sense that your claim about exceptions is true: that's the most common use for the print statement. But if someone writes a simple unix utility that reads from stdin and writes to stdout and they use the print statement to produce the output, would they be wrong? Or would they be using a language feature in a perfectly reasonable way which was understandable (once you knew about it) and had no performance problems?
Imagine someone saying "the + and * operators exist to perform addition and multiplication on integers". It is mostly true, that's the most common use for these operators. But if someone wrote a bitset class and implemented some of the features using + and * operators, would they be wrong? Or would they be using a mathematical feature in a perfectly reasonable way which was confusing to people who have never done bit-level manipulation (unless they included clear comments) and had excellent performance?
What I'm trying to say is that I'll listen if you argue "It is confusing to the reader for the following reason...". I will listen if you argue "It has the following performance issues...". I will listen if you claim "It can be incorrect in the following corner cases...". But when you say "It is bad because you are not intended to do it..." I will scoff and wonder who you think makes these rules.
http://xach.com/rpw3/articles/0dudnfoyYfsNu8vanZ2dnUVZ_sKqnZ...
http://www.nhplace.com/kent/Papers/Condition-Handling-2001.h...
http://people.csail.mit.edu/gregs/ll1-discuss-archive-html/m...
An early version of the Dylan manual had a good chapter on this subject:
http://jim.studt.net/dirm/interim-66.html
And Dylan has some material available as well:
http://opendylan.org/books/dpg/exceptions.html
http://opendylan.org/books/drm/Conditions_Background
http://opendylan.org/documentation/intro-dylan/conditions.ht...
But it doesn't mean that abusing exceptions as control structures is a reasonable idea.
In JavaScript, you have the ability to break to a label to escape nested loops. That's a limited form of a non-local exit.
Ola Bini was working with a condition system in his experimental / hobby language Ioke as well. These ideas get around.
You'd probably be surprised at how some of this works in Smalltalk as well.
"Exceptional" and "unanticipated" are not synonymous.
Edit:
I would add that it is true that there is a semantic burden with exceptions in that they are suggestive of error, rather than special case, conditions. So I would be very wary of using them in that way simply in my own code because it will potentially be misleading to consumers of the source code.
But if the clarity of the remaining code is sufficiently enhanced then it's a fair trade-off.
That depends very much on the idioms and conventions of whichever language you’re using. In Python, for example, there are some built-in exceptions like StopIteration that are used for routine flow control and do not in general represent an error.
Keep doing your own thing.
Happy days!
Also how should a a mathematical proof that code that uses exceptions for jumping out of iteration doesn't confuse anybody look like? It's just confusing because it's never done like that. It confuses programmers. At least one. QED?
You CAN devise your own coding style, but when using the conventions reader can safely assume couple of things. Reading code that doesn't follow these conventions is more difficult (for everybody except you).
No it’s not. Be sure to clean up anything that could need cleaning up in a finally block.
This is like saying... be sure to never copy more bytes than the buffer capacity. Easier said than done.
Writing exception safe code is very hard. Do not take my world for it. Read Alessandro Warth's paper (with Alan Kay as a co-author) [1]. Do not skip section 3...
Let me quote section 3.1:
In languages that support exception-handling mechanisms (e.g., the try/catch statement), a piece of code is said to be exception-safe if it guarantees not to leave the program in an inconsistent state when an exception is thrown. Writing exception-safe code is a tall order, as we illustrate with the following example:
try {
for (var idx = 0; idx < xs.length; idx++)
xs[idx].update();
} catch (e) {
// ...
}
Now, I have seen a lot of code, and very very very few times I've seen someone restoring the state of a collection after an exception blows.
The authors of the paper propose their "worlds" API as a way to make writing safe code easier. They're still using exceptions, but they would require no cleanup...
The code becomes:
try {
in thisWorld.sprout() {
for (var idx = 0; idx < xs.length; idx++)
xs[idx].update();
thisWorld.commit();
}
} catch (e) {
// no clean-up required!
}
This is a case where the developer needs to know that all update calls may not be completed. This may be a high or low probability, and may have fatal or no consequence. How many programs can handle a hard drive crashing or CPU glitch? Writing good error handling requires a lot more than removing exceptions, and often is not worth the cost. No software handles every error condition.
You can dumb it down as much as you want, people are still going to screw it up.
And you have exactly the same issue with "classical" error handling: you have to undo the partial job. Here's a bit of C code I wrote just this morning which closely ressembles your example:
for (i = 0; i < pdata->overlay_nr; i++)
if ((ret = overlay_init(data, i))) {
while (i--)
overlay_destroy(data, i);
goto overlay_err;
}
I use exception quite a bit in C++, but it does fit my coding style. I use RAII almost exclusively, which alleviates most of the issues. That means that I don't have try{}catches everywhere, mostly only when I want to do actual error handling.
Exceptions are a tool, it's up to you to use it correctly (or not use it).
Your example illustrates this nicely: say you jumped on the ideological bandwagon and decided that exceptions are evil and must be avoided. How does this change your code? You can try C / golang style where errors must explicitly be checked & routed through normal flow control - something like this has a million variants floating around:
var recover = 0
for (var idx = 0; idx < xs.length; idx++)
err = xs[idx].update()
if err:
recover = 1
break
}
if recover > 0:
// TODO: hard part goes here
It's people blaming their golf clubs for not being good enough, when the real problem is they need to learn to swing properly.
A solved problem except when people insist on using approaches where it's not a solved problem? I couldn't agree more! :-)
firstly, the update example is a bait and switch. Partially updating an array is not inconsistent. It may be wrong depending on what the contract of the function says but that's but the same as inconsistent.
Indeed the example that prompted this article back when I wrote it many moons ago was a case where you wouldn't want the state restored
> In general, however, this is not sufficient since update may also have modified global variables and other objects on the heap
"Doctor! It hurts when I do this!" "Well don't do that then"
It's hard to write any correct code when you're mutating globals and arguments all over the place. Exceptions can certainly make this worse.
In general if you're not using globals or mutating your arguments this isn't a hard problem. If you are mutating arguments and you want to ensure a "restore all state if an exception is thrown behaviour then yes this is sometimes hard (unless your argument supports a rollback mechanism - e.g. it's a database) but I submit that a large part of why you haven't seen that is that it's a contract people don't care enough about to support
This means that going against the grain will cost you time (and not you necessarily, but your company / colleagues, etc), and that time better come with some great benefits for it to be worthwhile.
So really, the argument can be looked at from many angles, depending on what value system you are using / what you would like to optimise for. I like to optimise for least surprises / development time.
My point is just that even if the exception-raising approach was better, it'd take a lot of time before it would become truly idiomatic and allow us to reap its benefits, on a software engineering level. I know many would disagree with me on this, however; I like to think I'm a realist.
Basically, you have nested validation code, and the second you hit something which invalidates your data you throw an InvalidDataException(X), catch it at the top of the validation, and then report back to the user that their input is broken because of X.
The alternative was that every method would need to pass back whether it had found a validation error, and every place that called one would need to check that returned value. Huge numbers of lines of code, for no real gain.
(Obviously, this doesn't work if you want to return _all_ the things that are wrong with the data.)
public Session startByInterviewId(Long interviewId, String email, String name) {
Session session;
Interview interview = interviewService.getPublicById(interviewId);
if (interview != null) {
Account user = userService.createUser(name, email, interview.getLocale());
if (user != null) {
session = startByInterview(interview, user);
}
}
return session;
}
public Session startByInterviewId(Long interviewId, String email, String name)
throws Interview.NotActive, Interview.NotPublic, Interview.DoesNotExist, Account.EmailExists {
Interview interview = interviewService.getPublicById(interviewId);
Account user = userService.createUser(name, email, interview.getLocale());
return startByInterview(interview, user);
}
Instead of just getting nulls to indicate failure I now even know exactly what caused my error.
I would never even think of writing code like this in JavaScript or Clojure though and I'd guess is that in Scala/Haskell a custom Option would be much better.
Actually: while the second code block is more readable, if I encountered it in the wild I would assume that it's full of bugs. It just doesn't seem very idiomatic. I'd much rather take advantage of the 'goto' nature of the return statement:
public Session startByInterviewId(Long interviewId, String email, String name) {
Interview interview = interviewService.getPublicById(interviewId);
if(interview == null) { return null; }
Account user = userService.createUser(name, email, interview.getLocale());
if(user == null) { return null; }
return startByInterview(interview, user);
}
But your point about the exceptions adding more information is well taken.
public Session startByInterviewId(Long interviewId, String email, String name) {
Interview interview = interviewService.getPublicById(interviewId);
if (interview == null) {
return null;
}
Account user = userService.createUser(name, email, interview.getLocale());
if (user == null) {
return null;
}
return startByInterview(interview, user);
}
Your code doesn't preserve information about what caused the failure though.
And also, I actually did magically reduce lines of code since I can just let the exceptions I fix bubble up and I've customized my web framework to turn them into appropriate error pages and the the <Class>.NotFound exceptions are even annotated so that they're automatically turned into 404s.
If you don't care about the specifics of the error then you catch that and handle it. If you do care then you're going to need to handle the specific cases anyway.
Personally I like checked exceptions at layer boundaries.
To properly understand condition handling, it is critical to understand that it is primarily about protocol, rather than mere computational ability. The establishment of protocols is a sort of before-the-fact hedge against the "prisoner's dilemma"; that is, it creates an obvious way for two people who are not directly communicating to structure independently developed code so that it works in a manner that remains coherent when such code is later combined.
This drives me mad in Visual Studio, if you specify a set of exceptions to break on, then later decide you need to catch all exceptions, there's no easy way to go back to the first set.
I'm surprised this question and partial answer on Stack Overflow haven't got more votes:
http://stackoverflow.com/questions/5452480/how-to-save-and-m...
http://www.javaspecialists.eu/archive/Issue187.html and http://onjava.com/pub/a/onjava/2003/11/19/exceptions.html?pa... are two well written sources of many that Google turned up on the subject.
http://jawspeak.com/2010/05/26/hotspot-caused-exceptions-to-...
The first link (http://www.javaspecialists.eu/archive/Issue187.html) says the code was hard to understand, that this is the 'wrong' way to use exceptions, that reusing a normal exception type instead of an exception intended for control-flow might mask other exceptions, and that debuggers will pause. These are MacIver's points #4, #2, #6, and #5 and he debunked each one.
The second link (http://onjava.com/pub/a/onjava/2003/11/19/exceptions.html?pa...) says "This not only makes the code difficult to read, but also makes it slower." That is MacIver's points #4 and #1 and he debunked them.
Do you feel that any of MacIver's points are incorrect? Do you know of any arguments that he did not already address?
He claims exceptions are fast enough. But though he proposes that exceptions are generally usable, he only backs this up for java, and AFAIK the statement is in general false. In other words, in most contexts, using exceptions for control flow is expensive.
He claims that "decent debuggers" won't pause - but which decent debuggers is he talking about? Because that's not at all the norm. Furthermore, as soon as you attach a debugger, it's likely all exceptions are much, much slower. Having code that runs orders of magnitudes slower in debug mode is a problem in and of itself because it means that some issues can't be properly debugged. Furthermore, many debuggers are designed to work with convential code - in other words, if they let you filter exceptions, it's not in a practical way intended to be ideally usable as control flow.
He claims that it's not hard to understand, but provides no support. If you're going to deviate from convention, you should have a reason; some argument. Merely the fact that it is conventional is a plus since that simplifies communication and maintenance. And frankly, having read his article, I don't see the advantage - he's doing it differently, but what's the point? What's he winning?
He claims that the arguments against exceptions aren't well documented, but that's simply because he's searching rather narrowly. All the way back to dijkstra's article on goto such control flow existed, it just wasn't called the same. And exceptions are essentially non-local goto's.
Finally, he's doing an end run around the type system. This is about as problematic as a null reference; i.e. it's going to cause constant pain. You've documented your program by type annotation which is good for humans, good for API discoverability (autocomplete) and allows some static checking. By doing this you're adding a hole that isn't properly annotated. In some languages, e.g. java with checked exceptions, this is less of a problem (but then it's also quite wordy). And again, it isn't true in general - maybe for java.
In a purely functional language with checked exceptions, I can see his argument holding water - Say, some Haskell variant. At that point, the distinction between this approach and discriminated unions is very small - indeed, if you use some nice monadic syntax sugar you could probably exchange the one for the other.
But really, why not just use discriminated unions in the first place?
This is going to vary across languages. In cPython for instance (since I happen to know it), exceptions are at least as fast as looping through an iterator, since exiting from a loop is done via exceptions. He included a link to an article showing that exceptions are quite fast on the JVM. I am sure there are other languages where the implementation of exceptions is slow -- in these languages this is a valid objection to using exceptions for flow control. And I don't care how fast the code runs under a debugger. Perhaps you do, but really, I don't; we must have very different use cases.
> He claims that it's not hard to understand, but provides no support. [...] I don't see the advantage - he's doing it differently, but what's the point? What's he winning?
It ISN'T hard to understand if used in a consistent way... I don't need any further evidence of that than my own observations. But you are absolutely correct that the argument needs evidence of exceptions being clearer. Perhaps it would help to provide an example of a triple-nested loop with numerous boolean condition variables contrasted with the same loop using exceptions for exit. I don't think exceptions for flow control are more clear in all circumstances, or even in most circumstances, but I believe that they are in some circumstances, especially in cases where substantial work needs to be done in order to determine whether one should proceed: doing the work twice is inefficient, and splitting the function into part-A (everything before the decision point) and part-B (everything after it) can be much less readable than using an exception if part-A sets up variables and data structures used in part-B.
> All the way back to dijkstra's article on goto such control flow existed, it just wasn't called the same. And exceptions are essentially non-local goto's.
No, I think they are fundamentally different. The problem with GOTO, and with programming before structured programming was invented, was a problem with arbitrary ENTRY points; arbitrary EXIT points do not cause this problem. My more detailed thoughts on this can be found at http://mcherm.com/permalinks/1/in-defense-of-the-much-malign...
> Finally, he's doing an end run around the type system.
This seems to be a completely new argument and one which wasn't addressed. I am very interested, but somewhat confused. How does the use of exceptions undermine the type system? Is it because the type of the function, instead of being "Returns an Int" becomes "Returns an Int or raises an exception"? If that happens for all functions, how is it confusing? Can you give me an example of how things could go wrong?
> In a purely functional language with checked exceptions [...] if you use some nice monadic syntax sugar [...] the distinction between this approach and discriminated unions is very small. [...] why not just use discriminated unions in the first place?
I agree, as long as the language syntax supports doing it easily that is the better approach.
There's no compelling reason to introduce odd, non-idiomatic code patterns when they don't confer a significant benefit. Some people here argue that the "intent" of exceptions in a language design is not relevant compared to what can be done with them. In general, I disagree with this. Language designers' intent informs the expectations of readers of code, and your readers' expectations are important. Code is a form of writing for communications, both with the computer and with maintainers; it's not artistic writing, where you might deliberately violate expectations or norms in order to delight or surprise your audience.
That said, using exceptions as a method of transferring control to a non-local, dynamically-bound location is a powerful technique for which few languages offer an explicit alternative. It's also difficult for readers to analyze, so it should be used sparingly. For instance, I can think of reasons that one might use this technique within a container or algorithm library, but I can't think of a good reason for a container library to force this control flow idiom on callers by throwing exceptions to signal common situations.
(As usual, Common Lisp is unusual here. It has a throw/catch feature which is explicitly a flow control mechanism, and not just an error-handling mechanism. The standard documentation is interesting in that the Notes section suggests exactly when one would want to use this mechanism: http://www.lispworks.com/documentation/HyperSpec/Body/s_thro...)
But I was probably not trying to suggest you should regularly do this (certainly I wouldn't suggest that now), only that if you find you have a good reason to do it you shouldn't be afraid of it.
The main thing I would add to the original article is that the author is being a little kind to the critics on some of those points. For example, not only do exceptions not inherently need to be slow, they can actually be faster than the “classical” equivalent via return codes and the like. This is essentially because any jump-table-based exception mechanism means the non-exceptional control path can omit all the test logic that would serve only to exit the current block early. That could both reduce the size of the non-exceptional code and reduce the number of potential branches, each of which can be helpful for optimization. Although it’s rather less significant in performance terms because it only tends to happen once, when an exception does occur you also don’t have to run through all the individual levels of unwinding the stack with a jump table approach, as you can jump straight to levels that actually need to do something on their way to handling the exception and skip over anything that was just going to return immediately anyway.
As another minor data point, I haven’t found exceptions in C++ to be slow. Most/all of the major compilers seemed to have moved to a table-driven approach last time I used a broad set of them. The nasty overhead in C++ is more likely to be in the size of the generated executables; I’ve seen a compilation with exceptions disabled shave nearly 1/3 off the executable size.
Of course exceptions have somewhat different meanings and can have very different implementation overheads in different languages, so I’m not arguing that all advice to use them sparingly is bad. As always, it’s important to separate the dogma from the rational arguments.
Exceptions are not suitable for control flow. Yes they work and can sometimes give nice features, but typically they are just showing that you have poorly designed code and should be doing something differently.
'/s/Exceptions/GOTO/g' and you have essentially the same argument
Absolutely untrue. Eliminating GOTO was a major step forward in programming language design, one better known as "Structured Programming". The advantage was that it eliminated random entry points so that bits of code ("subroutines", "methods", or "functions") could be reasoned about independent of the entire program by starting from the "top" and working forward. Arbitrary exit points (like exceptions, break, and return) still permit this reasoning; allowing GOTO to target arbitrary statements makes it impossible to reason this way.
For a better explanation, see this essay I wrote on the subject: http://mcherm.com/permalinks/1/in-defense-of-the-much-malign...
This is why java implemented typed exception throwing because it the only way to have some semblance of safety when programming with exceptions. I am even more disheartened to see that the author uses my beloved scala and in still trying to do this kind of nonsense, when we have plenty of good tools at our disposal to negate the use of exceptions for control flow.
I think that RAISING exceptions is just a non-local exit point, like a break or return which does not make it difficult to reason about one piece of code in isolation from others.
I think that CATCHING exceptions is a clearly marked 'entry' point: one can only arrive in an exception handler from an exception raised within the corresponding try statement, and the place that you arrive is clearly marked.
The one bit that is more confusing is throwing right through without an exception handler: in other words, code you call might raise an exception which your code does NOT catch, and thus exit your function prematurely: you might thus allocate some resources and fail to deallocate them safely.
Are we saying the same thing here?
Let me try to give some examples. I write things like this:
Person p = getThePerson(id);
try {
storePersonToDatabase(p);
} catch(DatabaseException err) {
generateAlert('Could not save person');
}
doMoreProcessing(p);
Person p = getPrimaryAccountHolder();
try {
storePersonToDatabase(p);
p = getSecondaryAccountHolder();
storePersonToDatabase(p);
} catch(DatabaseException err) {
generateAlert('Could not save person');
}
doMoreProcessing(p);
Person p = getPrimaryAccountHolder();
try {
storePersonToDatabase(p);
Person p2 = getSecondaryAccountHolder();
storePersonToDatabase(p2);
} catch(DatabaseException err) {
generateAlert('Could not save person');
}
doMoreProcessing(p);
If I didn't follow this practice, then I think the problem you describe would be an issue for me: not knowing which of several possible locations was the source of the exception would mean I didn't know how the variables had been set. Interesting... I learned something today.
However they are only few good reasons to use goto, and exceptions for control flow should be treated the same.
Therefore the point we should make against exceptions for control flow is not exceptions are for exceptional circunstances but; as a rule of thumb exceptions are for errors. And indeed they help a lot in not obscuring normal program logic, we allready have other tools for control flow.
Nobody likes goto (and my bad english)
bye
In Objective C, exceptions are "he's dead Jim" territory.
Some of the differences of use are caused by the implementation of exceptions in those environments and the expectations programmers of libraries in those environments to what happens when exceptions occur.
Sadly dead once the author got his PhD though.
Blocks exist in Objective C. GC did, but was replaced for Automatic Reference Counting (ARC).
ARC is good enough in mobile and probably better for Desktop.
> Many years ago
> (before Objective C had exceptions or blocks or GC or ...)
TOM had the things that I mentioned back in 1998, 1999, maybe earlier.
Objective C has come a long way since then.
ex. Lets assume that a problem can be attacked as algorithm1 (A1), and algorithm2 (A2). A1 is fast but cannot handle some corner cases while A2 is slow and can handle all cases. we also assume that there is no easy way to tell whether A1 is good or not without invoking A1.
So the function can be implemented as:
void A()
{
try {
A1();
}
catch (e) {
A2();
}
void A1()
{
.....
if(corner case) raise();
.....
Is there a simple way to avoid using eceptions here?
bool A1();
bool A2();
void A(){
bool result = A1();
if (!result){
A2();
}
}
However, it becomes more difficult if you want a return value. Suppose that your A() function was as follows.
double A(){
double result;
try{
result = A1();
catch (e) {
result = A2();
}
return result;
}
double dist(){
double output = dist1();
if (output==-1)
output = dist2();
return output;
}
bool A1(double* output);
bool A2(double* output);
double A(){
double output;
bool success = A1(&output);
if (!success)
A2(&output);
return output;
}
Because programming is just as much about communicating intent as it is about writing efficient code! Following guidelines and maxims is extremely useful because if both the writer and the reader understands the protocl, then communication is smoother.
The "Exceptions are for exceptional circumstances" protocol means that the reader of the code doesn't have to wonder "Is this a control flow situation? Is this exception supposed to be handled? Where does the code resume?"
Following the Samurai Principle (http://c2.com/cgi/wiki?SamuraiPrinciple) similarily eases the cognitive load of the person reading the code. It allows you to treat each callable piece of code as an isolated unit whose value is equal to what it returns. For non-critical code, you can completely forget about exception handling because you dont need to bother. However, if an exception is thrown you can be equally assured that something unexpected happened. It was to hard for the "samurai function" to handle. Simple and extremely convenient.
One example of not following the Samurai Principle or "exceptions for exceptional situations" happened at my last job. We were an mobile payment processor and trying to issue a charge to a customer when their balance was to low would result in a BalanceToLowException. Except that wasn't anything out of the ordinary, customers would often not have enough funds! The situation would be handled by retrying the transaction some other day. The API for charging customers would better have been designed returning a two tuple (chargeStatus,errorMsg) so that the following code could continue in the same location whether the charge succeeded or not. Throwing exceptions for ordinary events lead to spaghetti code.
You can see similar misdesigns in some database ORM:s in which some queries throw silly NoRecordsFoundException which user code is forced to handle. Or Python's classic
try: return int(somestr)
except ValueError: return None