An Introduction to Condition Handling for Non-Lispers | Hacker News Reader

An Introduction to Condition Handling for Non-Lispers

lubutu.com

104 pointslubutu65 comments

> why don’t more languages have them?

My guess would be that they are not needed. Exceptions for anything other than program death are something of an anti-pattern, potentially spreading the internal details of some piece of code all the way up the stack.

My choice of implementation would be to pass a discriminator function which takes an error and returns true if the error is recoverable. Then you have the choice of either returning all the errors at the end or reporting each earlier when the discriminator is called.

Edit: That's not to say that exceptions can't be used effectively, they just mostly aren't.

> My choice of implementation would be to pass a discriminator function which takes an error and returns true if the error is recoverable.

Conditions are close to exactly that + syntactic sugar.

`handler-bind` lets you establish a dynamic scope, inside of which specific functions are called when specific conditions are signaled. Signaling a condition = call the first handler function in the list that handles the signaled condition. The handler is called before the stack is unwound, which allows it to elect to either `throw`, invoke a restart, or ignore the condition entirely.

This is in contrast to exceptions in (say) Java. `throw new FooException()` does a lot of potentially destructive work before control gets transfered to the exception handler: It allocates an exception object, which fully captures the current thread's stack. (It's not too common, but this can be bypassed, if you don't throw a new exception object...) It unwinds the stack to the frame that has a matching `catch`.

Only after those two things have happened does the exception handler get a chance to decide how to proceed, and at that point, it's too late for many recovery strategies. Conditions help this by making it easier to intercept the process earlier.

Where they hurt is that they add another level of complexity to API design. With Java style exceptions, if the function fails, it fails completely. There's a convenient simplicity to that. With Lisp style conditions and restarts, A function will let you know how things are going as it does its work and then give you the opportunity to change its behavior midstream. It's a lot more work to design that kind of API, and it's probably well past the point of diminishing returns for most kinds of development.

Great answer. I'm left wondering if the solution is more complex than the problem.

There might be two different perspectives on this:

* automated error handling, mostly invisible to the user. In this case the condition system may give an attractive architecture, but it is not widely explored and it is not clear how it would integrate into a language like Java. There could be a benefit, but it is not explored by software architects. Apple's Dylan probably had something to it, but it was not widely used.

* error handling in interactive systems. That's where the Lisp community used/uses it mostly. The largest effort to use the condition system were in the OS of the MIT Lisp machine descendants - especially the one from Symbolics. It was used both in the development environment - which is tightly integrated in the OS. Most of the development tools were using it. But things like the networking code or the file system were also using it. The user interface is generally slightly more complex than current graphical user interfaces and it was expected that the user used this kind of power. When you get a dialog of restarts presented, the user needs more thinking about how to proceed than with a simple cancel/abort dialog box. There were also applications written using it and for that slightly simpler interfaces were used. For example the restarts were presented in dialogs and not in a debugger context.

It takes a bit of re-orientation to imagine a user interface, where the user has the option to repair/retry a failed operation, instead of the usual abort/redo. These benefits are easier to get in an integrated system (-> Lisp, Smalltalk) - a model where several non-integrated tools interact in a condition system is more difficult to imagine.

Today, most of the better Common Lisp systems are using the condition system in their environment. There the power can be used, but at the same time it is also easy to ignore and just use a default mode of using the abort restart - without getting the benefits of thinking a bit more and investing the time to use a more complex restart.

In practice the option to resume is used much less frequently than the option to throw. There are two primitives one can use to signal a condition: SIGNAL allows resumption, but ERROR does not. So if you don't want to deal with the complexities of a possible resumption, simply call ERROR.

Similarly, the client, to establish the handler, can use HANDLER-BIND, which is the fully general form that permits resumption, or HANDLER-CASE, which is simpler to use but does not permit resumption.

In practice, most of the time people use ERROR and HANDLER-CASE, and so the extra complexity does not intrude at all.

OTOH it is occasionally handy to be able to resume.

My understanding of Seibel is that Lisp's condition system is designed to avoid exactly the problem of internal details spreading up the stack.

It does so by allowing conditions (of which exceptions are one type) to be handled by a higher level of the call stack without unwinding the call stack, and allows for the determination of which particular condition handler to be made independently (and at a higher level of the call stack) than the implementation of the handler.

The "anti-pattern" may be a result of the way in which other languages implementation of exception (or condition) handling differs from that of Lisp.

> [conditions are] handled by a higher level of the call stack without unwinding the call stack, and allow for the determination of which particular condition handler to be made independently (and at a higher level of the call stack) than the implementation of the handler.

Is exactly what the higher-order function I described would let you do. Why do we need conditions?

Edit: the answer to this question is "because we want to use exceptions". But there's really no need to.

> Is exactly what the higher-order function I described would let you do. Why do we need conditions?

Because conditions let you do that correctly without e.g. changing all intermediate pieces of code so they forward some sort of gigantic mapping of callables through the stack.

A possible benefit of conditions in Common Lisp is more than 25 years worth of implementation experience and full documentation of how the system is structured.

Extensive literature about the language is a defining feature of Lisp.

Conditionals as detailed in Common Lisp the Language, 2nd Edition: http://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node312.html#S...

Hmm, the same argument could be made of GOTOs in FORTRAN.

GOTO's use in Fortran has continued to be deprecated for several decades.

One might say that they are not Fortrany in the way that conditionals are Lispy.

That's non-essential, but certainly nice to have. Thanks for the insight.

That is, of course, the standard argument for dynamic binding in general.

So in CL one could certainly pass a callback via dynamic binding and get the same effect.

The choice to use the condition primitives instead is, then, simply a stylistic one.

For those who wonder whether that is worth a new idiom to learn: the stylistic effect is enormous.

For example, there could be 5 layers of code between a condition handler and the code signaling a "file not found" condition, and the top levels of that stack might not even be aware that they could trigger such a condition or even any condition at all.

In some sense, this is the reverse of Java's checked exceptions. There, you have the problem that an interface declares exception X and Y, and an implementation that wants to throw a Z has to encapsulate that exception into a X or an Y. Here, the intermediate layers _can_ be blightfully unaware of those conditions.

A common use case is to use conditions without registered handlers as "break on exception" breakpoints, with the benefit that the human working with the debugger has more options than "stop execution", "continue and pray" and "tweak some memory, set the PC, go, and hope for the best". For example, if the 'open file' system call has a restart, users can easily fix "oops, that lengthy computation is done, but I mistyped that output file name." error conditions.

Per Greenspun's Tenth Rule you do not.

http://paulgraham.com/quotes.html

Lisp's condition system allows you to handle the condition without unwinding the stack.

That's the whole point.

So does a higher-order function...

Therefore it can't be the whole point, because that mechanism already exists, which most commenters here seem to have not realised.

The actual point is, that it gives you more powerful exceptions, but you don't actually need them.

Not the same, because you need to pass function to be called on error throught all the stack, and you don't need to do it with conditions, if I understand correctly.

Example: main calls calculateSth calls f1 calls f2 calls f3 calls readFiles

If we want to decide in main how we want to react to errors when reading files, we need to pass the function to handle errors thorught the whole chain.

With conditions calculateSth,f1,f2 and f3 don't need to know about our condition.

This also means that when you add new condition to function that you invoke in many places, you need much less changes everywhere.

At last, an actual answer, thanks. Basically, we can save the need to pass a callback argument.

This reductionist philosophy is mostly inappropriate when talking about programming language features.

Yes, from what I understand about Conditions (IANACL), they can be implemented with first class functions and dynamic variables. And dynamic variables can be implemented by passing an extra argument to every function. And function arguments can be implemented with well-named global variables. And functions can be implemented with computed gotos and a reified global stack variable.

But the point of a language is to create a common vocabulary with which programmers can think and talk to machines and each other. Try to reduce everything to a simpler form and everyone ends up reimplementing the higher level concepts in incomplete or at least incompatible ways with everyone else. There is a need for experimentation and turnover to figure out which higher level constructs are best, but this is entering the realm of language design and should not be a required part of application programming.

It's not even that, this can be (and probably is) implemented with dynamic variables (or some similar mechanism). You can also implement exceptions with setjmp and longjmp in C.

I feel the most important thing is conventions, and having a weaker model of exceptions as a convention, affects all/most code in the platform. In Java most libraries throw exceptions and so you will probably use exceptions. In C most libraries return an int return value for error/success and you'll probably use that. In Common Lisp, people use conditions.

So, what I'm trying to say, is that when considering this kind of design decisions in a language, we have to take into account how it affects the whole platform (and not just see if it is possible to implement it any other way in our particular language of choice). The same reasoning should be applied for object orientated features, scoping rules, calling conventions, multiple-value returns, lazy evaluation, macros, continuations, modules, type systems, naming conventions, etc.

And the potential need to pass way more than one, if you have 5 different error conditions which can exist in the same stack you need to come up with some sort of standard way to express that (e.g. a mapping of condition to callable). Then you need to find out how to express inheritances/overrides, and the like.

Nah, you only need one error callback with a parameter, i.e. `f(error)`. You could override this at any point using closures. We're talking almost identical code.

And you also define the recovery strategies within the function that cares about them. It makes the code much more readable.

You also continue the computation from that call-site without losing the stack. If after a particular restart is invoked, processing can continue along the chain without missing a beat.

No... we could already do that with higher-order functions. We're going backwards here, please read the previous comments, specifically from ajuc.

As stated, a higher order function gives you all of these features already. (The discriminator function I suggested would just return a bool to indicate whether or not error recovery should occuur, which is handled at the call site, in the appropriate context, with the appropriate stack).

All Condition Handling brings to the table is the ability to omit passing the callback around. All the other features were already there.

Edit: mschaef's comment confirms this.

Peter Seibel's explanation in the online version of Practical Common Lisp Chapter 19:

http://gigamonkeys.com/book/beyond-exception-handling-condit...

Practical Common Lisp Table of Contents: http://gigamonkeys.com/book/

For anyone that had trouble learning the condition system, or just wants another example, this helped me:

http://symbo1ics.com/blog/?p=1405

That task here is: A 'compute-slopes' function gets a list of points and returns a list of slops between consecutive points.

The problem: What if points have the same x-value, hence the slope is undefined.

Solution 1 Exceptions: throw new DivisionByZero()

Solution 2 Condition Handling: provide the restarts point to (a) return nil, (b) return 0, and (c) return a user-specified value.

Solution 3 Return Error: return Error("division by zero")

What I do not like about this: You expose implementation details to the outside with those recover points. In the same sense exceptions in Java expose implementation details, though.

For this specific example, the better behavior (imho) would be that parseEntry always returns. So it returns ParseError instead of throwing it.

Basically, I want a more convincing example for condition handling.

> You expose implementation details to the outside with those recover points.

Of course not, no more than an exception or an error code "exposes implementation details". Condition restarts are part of a library's API, since they're a way for library users to interact with the library (in this case, to customize the handling of some error conditions)

> For this specific example, the better behavior (imho) would be that parseEntry always returns. So it returns ParseError instead of throwing it.

How and why is it a better behavior, especially if — unlike this contrived example — there are 5 frames of foreign code between the caller and the condition being signalled? Return or exception will have to unwind those frames, the point of a condition is that you don't, instead right there the library can provide a hook through which the library user can respond to the question: "how do I handle [this error]?"

That's one idea of the Lisp condition system: you don't return. The handler gets called on error and the full context is still there. The context includes all bindings, dynamic handlers and all possible restarts. The restarts itself are most often generic: Abort, Continue, Use Value, Store Value, ... There can also be several of those. For example the log error could be restarted by using a Abort error to abort a specific action (say, a compilation of a file) , to abort the whole process (say, a compilation of a whole system).

My favored method: return an error object, which includes all the necessary context. Of course, the library writer must anticipate what information might be necessary, but with condition handling he also has to anticipate, what restarts are necessary.

if you return, you can't continue.

With the condition system you implement restarts as a service to the user. You don't have to provide any one. In a typical Lisp system there will be a default handler.

The more restarts you implement, the more choice the user may gets. The handler can also decide which and how many restarts it wants to expose to the user, if at all.

> My favored method: return an error object

Right, so you've unwound your whole stack and now the original caller has to re-set the initial conditions (hoping all the state was discarded with the stack and none leaked), alter whatever needs to be changed (assuming and hoping there's even a way to get a handle on that) and re-wind the whole stack from scratch.

> Of course, the library writer must anticipate what information might be necessary, but with condition handling he also has to anticipate, what restarts are necessary.

Just as you have to "include all necessary context" in your solution...

Suppose we were attempting to write parseLogInteractively by just returning errors (as in idiomatic Go).

If parseLog just returned an error upon failure we would have to same problem as exceptions do, as explained in section 1. So we have to make parseLog return two lists: one of entries, and one of errors. We would then have to search through the error list fixing and reparsing each error. In Python this would be something like,

  def parseLogInteractively(file):
      return parseLinesInteractively(file.readLines())

  def parseLinesInteractively(lines):
      entries, errors = parseLines(lines)
      if errors:
          entries += parseLinesInteractively(askToFixEntry(e.text) for e in errors)
      return entries

Again we lose the ability to abstract the function 'parseLog'. And I don't know about you, but this looks far worse to me than "resume FixEntry" in terms of exposing implementation details.

In python, I usually deal with this with generators. It's not as elegant as conditions, but it works well enough. I've never tried to permit custom restart-like behavior, but now that generators are coroutines, it should be doable.

e.g.

    def parseEntry(line):
        try:
            parsed = foo(line)
            yield parsed
        except SomethingBad:
            yield None

Common lisp could really use some better coroutine support (and some way to specify generic sequences). Sure, you can do it with macros, but it gets horrible fast. Have you looked at SERIES, for example?

> I've never tried to permit custom restart-like behavior, but now that generators are coroutines, it should be doable.

It would be extremely awkward, you'd need a big dispatch table handling the generator's result and send()ing restarts up to the generator, and of course you lose condition's ability to work through stacks of blissfully unaware code, and to have default behaviors (in Smalltalk, the default behavior — in an interactive image — is to pop up a dialog asking the user if he wants to unwind the stack [as in an exception], to open a debugger at condition point or to try resuming the condition [under the assumption that the user fixed some incorrect method, or checked his network access])

Thanks for the pointer; I've been learning some Smalltalk lately, but I haven't yet looked into the condition system.

And, yes, I'm not saying that manually implementing pseudo-restarts is something I'd ever want to do, but some vague skeleton of the possibility is there.

If you want the restart, you need to take advantage of the PEP-342 two-way yield(), as in:

    ...
    except SomethingBad:
        yield None 
        restartRequest = yield None
        if restartRequest:
            do whatever is necessary to go on

The problem with the above is that the double yield() ugly. You cannot do this with single yield() because the caller would have to know in advance that next() will return None.

Maybe something like: https://gist.github.com/3598432

although perhaps passing a 'handlers' object down, with which handler functions could be registered might be better?

Generic sequences are coming to common lisp. I'm working on a simple implementation inspired by clojure which I'll be posting to /r/lisp in the coming week.

Do not separate errors and entries. Instead use something like Haskell's Maybe, but instead of an empty Nothing-object you return something which contains error information:

  def parseLogInteractively(file):
    entries = list()
    for line in file:
      entry = parseEntry(line)
      if entry.failed:
        entry = askToFixEntry(entry)
      entries.append(entry)
    return entries

A design question arises: Do you need to fix parse errors before you can proceed with next entry? If not, the error handling can be done by whomever calls parseLog.

The abstraction problem remains: you are reimplementing parseLog every time you write a new handler, as I described in section 1.

(By the way, you're looking for Haskell's Either type, but ([a],[b]) and [Either a b] are equivalent here.)

No, you only implement this one:

  def parseLog(file):
    list = []
    for line in file:
      list.append(parseEntry(line))
    return list

Or better (more pythonic imho):

  def parseLog(file):
    for line in file:
      yield parseEntry(line)

The second version is lazy (if file reading is lazy), so you can even abort parsing or fix the state on an error. Here are your different versions all reusing parseLog from above and equivalent line count to condition handling:

  def parseLogLoudly(file):
    for entry in parseLog(file):
      if entry.failed:
        print "Warning:", entry.message
      else
        yield entry

  def parseLogSilently(file):
    for entry in parseLog(file):
      if not entry.failed:
        yield entry

  def parseLogInteractively(file):
    for entry in parseLog(file):
      if entry.failed:
        yield askToFixEntry(entry)
      else
        yield entry

(Haskell's Either has "Left a" or "Right b". I want something like "Just value" or "Error msg", but you are right Either is nearer than Maybe)

Haskell's Either type is conventionally used exactly like that--pretend that Right val is Just val and Left err is Error err.

The reason it doesn't have those names is because it is more general; you can also use Either to model things like early termination. The generic names just make it clearer that it isn't exclusively for error handling unlike exceptions. In other languages using exceptions for more generic control flow is considered poor style, but in Haskell using Either for non-error cases is completely reasonable.

(It's conventional to use Left for errors and Right for successes.)

Sounds like this is just an opaque way of providing a strategy or callback to the parseLog method. Why not something along the lines of (scala):

  def parseLogInteractively(file) = parseLines(file.readLines(), error => askToFixEntry(e.txt))

  def parseLines(lines, errorHandler) = lines flatMap {
    tryParseLine(_) fold {success(l) => Some(l)} {error => errorHandler(error)}
  }

> Why not something along the lines of (scala):

Because that only works for very flat stacks, otherwise your callback will start infecting more and more method less and less related to what's actually causing the error, and then you'll have to move to a mapping of callbacks as different parts of the stack will want their own error callback.

Do you really want to handle error conditions in a resumey way far further up the stack? I'd expect most cases are either handle the error close to the cause, or a single application-wide handler (which could be provided with dependency injection).

The condition handling approach just seems too magic to me. It seems to mean the "inner" code calls another method defined somewhere else - but that "somewhere else" is defined by the call stack, something that's not really very visible in the code.

> Do you really want to handle error conditions in a resumey way far further up the stack?

"Way far" is relative and depends on a number of factor. And I don't consider half a dozen stack frames "way far", it's very easy to attain when using a coding style emphasizing small short methods which call one another.

You also have no bloody idea how the library works internally.

> I'd expect most cases are either handle the error close to the cause

Again, "close" and "far" is relative. I want to handle the error at the closest point to its triggering, but I'm also limited by the data available to know how to handle it. One pushes the handler down the stack, the other up.

> The condition handling approach just seems too magic to me.

There's nothing magic to it.

> It seems to mean the "inner" code calls another method defined somewhere else - but that "somewhere else" is defined by the call stack

Welcome to dynamic scoping, there are cases where it's useful.

> something that's not really very visible in the code.

How is "I'm signaling a condition" not very visible in the code? It says right there that it signals a condition.

"The condition handling approach just seems too magic to me."

They're nothing compared to Lisp's macro facility.

I'm not exactly a fan of that either

Letting the caller do the handling and giving it access to the dynamic environment where the condition was signaled is a feature.

This paper by Pitman helps to understand the design choices www.nhplace.com/kent/Papers/Condition-Handling-2001.html

Controversial programming opinion of the day: not knowing implementation details will come back to bite you, overexposure is better than underexposure.

Anyway, I think the basic idea is just a small extension of being able to pause the program counter, change stuff, do something else, and at one's option continue where one left off, all without requiring a debugger. Nothing new if you've been exposed to the Lisp world. Typical examples of utility being recovering from longComputation() then failing without repeating work and hot-fixing a long-running server process. I agree this example isn't a very good one, I like your design you commented on for this problem better, I think in this case even "an error in parsing a line is not exceptional" would suffice in defeating the example. (Though in Python even syntax errors are implemented with exceptions, so...)

How would you accomplish somthing similar without exposing implementation details? It seems to me "this function may throw these conditions" is more a documentation issue. I think every other system I've seen, (esp C) exposes more details.

If I understand correctly, the idea is that `parseLog` is a library function, while `parseLogSilently` and `parseLogInteractively` are examples of callers to that library.

If so, then the complexity of documentation and understanding of `parseLog` is pretty high. The caller needs to not only know the different conditions (e.g., `ParseError`), but also the available recovery strategies for those conditions (e.g., `SkipEntry`, `FixEntry`), and further that some of those strategies require an additional caller-provided function (e.g., `askToFixEntry`) with its own separate requirements (e.g., it takes an exception and returns a line).

Why would it not be better, since there are a finite number of provided recovery strategies, to simply hide that complexity within library-provided functions? E.g.:

    (defn parse-log [file] ...) ; aborts on ParseError
    (defn parse-log-skip [file] ...) ; skips bad lines
    (defn parse-log-fix [file fix-fn] ...) ; emits (fix-fn line) for bad lines

Notice 'parseLogLoudly'. You could not implement that using your set of functions: you are not allowing for other code besides 'resume' inside the 'handle' blocks.

Also, I don't believe the mass duplication of code necessary for your approach is acceptable, even within a library.

>Notice 'parseLogLoudly'. You could not implement that

Fair point, and that's due to my failure in not providing an optional side-effect function arg to `parse-log-skip`. And this brings up a related point, namely that the author of the lib is still responsible for crafting the API. For example, if the author of `parseLog` had not included `recover FixEntry`, would it be possible for the caller to implement it via a condition system? Would that even be a good idea? Using the pseudo-python, the only approach I can think of would be:

    def parseLogInteractively(file):
      do:
        return parseLog(file)
      handle ParseError, e:
        line = askToFixEntry(e.text)
        retry

That reference to `line` is pretty horrific.

Perhaps this is one of those cases where a simplified example helps convey the concept, but is too simple to show where other approaches would fall short.

>mass duplication of code

I'm not clear on what you mean. Multiple functions with different behaviour seems not to altogether different from one function with implicit switches that change its behaviour.

Seems like useful feature to have.

Is it supposed to be only used for error handling (like exceptions) or can you use it for unconventional control flow, too?

Dan Weinreb (one of the Multics users who brought the Multics condition system to Lisp) wrote about why "error" is not a great term: http://danweinreb.org/blog/what-conditions-exceptions-are-re...

Personally, I've not seen it used for arbitrary control flow. If you control the syntax of things inside your block, then macros are probably good enough. If you're passing values outside of your own block, then you probably need continuations.

OTOH, Lisp programmers have a tendency to not care what something is "supposed to be used for". Using Lisp at all is said to be "the most intelligent way to misuse a computer". :-)

You can use it for unconventional control flows if you wish to. For instance, see [0] which implements dynamically scoped variables on top of conditions. I'm not sure that's overly common, but I'm a smalltalk dilettante

[0] http://www.lshift.net/blog/2012/06/27/resumable-exceptions-c...

I think you are looking for the "with" statement! The feature you describe is already there. And it works with objects, so it's reusable, extendable and exchangeable if needed.

http://effbot.org/zone/python-with-statement.htm

No, context management is only a small subset of what you can do with condition (and does not involve restart), and also implies stack unwinding (as do exceptions) amongst other things.

To TFAA: to talk about conditions but not use lisp syntax, you could have used Smalltalk (the other language with conditions and restarts)

We are not talking "condition" from Lisp here, but the blog post that is linked to this thread. And what the author is doing is asking for a "with" statement without knowing that the feature is already there. "condition" might be a powerful feature and I might even like it myself, but the post doesn't show the power of "condition" to Pythoners, because the feature used in the example already exists in the core language.

> We are not talking "condition" from Lisp here

Of course we are.

> And what the author is doing is asking for a "with" statement

No, he most definitely is not, a with statement (or Common Lisp's version, `unwind-protect`) does not provide resuming, only restoring a previously-stored context.

> the post doesn't show the power of "condition" to Pythoners, because the feature used in the example already exists in the core language.

You should try reading the post again rather than stop at the first paragraph, because you're wrong.

Reply on news.ycombinator.com