Pitfalls of Callback-Based APIs | Hacker News Reader

Pitfalls of Callback-Based APIs

thelig.ht

61 pointsrian39 comments

I think not just callbacks, which are currently a focus due to promise love, have many balancing ranges that programmers have to operate in to prevent technical debt and spaghetti code.

That is why programming is hard. Programming takes thought, design, planning, and why programmers aren't a factory line where you can just add a bunch in. Programming is solving tough problems and making them easier using good balance for time, budget, maintenance, integration and more.

Careful nuances, balances, structures, flow and many other things can be disrupted if programming is treated incorrectly or with lack of care from too many cooks, too much pressure, or not enough prototyping. This problem will always exist, good individuals and teams will always know there is no all encompassing fix from some new tech, there will always be the balance that determines success using all advancements at your disposal that help you and consistently improving that. The design has to continually keep up with the evolution of the project.

This article reads like a mind dump lacking a cohesive narrative... I mean look at how the author closes the article:

> This article isn't really about callbacks and it isn't even really about APIs.

Well then, perhaps you should have named the article something else...?

I thought it was a good narrative, stepping through the problem, the ending with the realization that there was no solution.

The essential problem is that any program which can incorporate arbitrary recursion is subject to the halting problem. The seemingly simple initial fixes, like a lock or asynchronous callback queue, can improve things. The author proposes third-party code analysis tools to help; I'm less sure of that. The Gödel dragon is not easily pushed back, and we often accidentally create Turing machines.

I think the title's fine. It's better than "Callbacks and the Halting Problem", since most of the essay covers, after all, "Pitfalls of Callback-Based APIs".

In this case, the dragon is slain without much trouble. In LVish (https://github.com/iu-parfunc/lvars) we've got an effect-tracking system that allows us to ensure determinism (read more here: http://www.cs.indiana.edu/~lkuper/papers/dissertation-draft-...). This would be pretty easily extended to only allow computations that are safe to log. And this is all in GHC Haskell! No external static analysis is necessary; the compiler can do it for us.

The dragon cannot be slain, only pushed back. It is the halting problem.

Yes, further automated analysis can find some of these problem. That's what the author of the essay proposes, and it's good to know that there are some solutions.

Let's go down the road further and expand the system so there are multiple loggers, and where subsystem logger configuration via an external configuration file occurs at run-time. Some configurations are acyclic, others are not.

This could be prohibited by edict - if a program cannot be analyzed then it's not valid. As the essay points out, there are also formal verification techniques - and that very few people use them. As I pointed out, it's surprisingly easy to make a Turing machine by accident. (See http://beza1e1.tuxen.de/articles/accidentally_turing_complet... and its HN comments https://news.ycombinator.com/item?id=6577671 .)

In practice then, it's very unlikely that most complex software will be able to use these tools, because it's hard to restrict the solution space to what those tools can analyze.

Here there be dragons.

I would title this article, "Infinite recursion considered harmful".

hehe or "Deadlocks considered harmful."

I find it impossible to take seriously any argument that claims to be true in general, but where the only example given is logging.

Logging is weird and special, running at unusual times in unusual states, and therefore has unusual requirements.

So, yeah, having a pile of arbitrary functions that your logger can call is not really going to work. But if this were a "notify when a comment is posted" example, the author would really struggle to find problems with that same approach, because that's a much safer operation that happens in a more predictable way than logging is.

The only real takeaway from this article is "be careful with your logging implementation," but hopefully you already knew that.

Not a very convincing example. A component provides a callback, and then the callback ends up using the same DB access mechanism that the component is using? Or the same lock? I mean, is it providing a callback to itself? Layering the code, or isolating the component from those who subscribe these unpredictable callbacks seem kind of obvious to me, and would prevent all of this. The author is right, though - there is something to take away from this, but I think it's simply the importance of clean separation of concerns. Or, the importance of good factoring, if I'm using the word correctly? Other than that, we're entering the realm of the eternal programmers' dream: programs that write themselves...

Mark S. Miller wrote his dissertation http://www.erights.org/talks/thesis/ in large part on expanding the "deferred callback" pattern this article recommends, including explaining how to make it applicable when it doesn't seem to be applicable. His solution is based on a sort of ultra-fundamentalist object-orientation approach, even though (as I pointed out in another thread) normal object-orientation makes these problems worse rather than better.

I don't think the "Pure Solution" mentioned in the article works. The only effect of a pure function is to compute its return value, so not only is it safe to invoke pure functions as callbacks, it's also perfectly useless — you can't tell whether it even got invoked or not. (Rian in another thread argues that qsort's comparison function is a "callback", but I don't think that's what people usually mean by "callbacks".)

Indeed, Miller's thesis is titled "Robust Composition", while this post says of itself it "isn't really about callbacks... [it's] about building robust systems." (And I was reminded of the thesis early on in reading this.)

Just curious, how would you describe what most people usually mean by "callbacks"?

I've always considered function arguments to functions like qsort() as callbacks, and from my interpretation of Wikipedia it seems to agree. Now I wonder if my interpretation has been overly general or if a more specific interpretation has become more commonplace.

While my impulse was to agree with you, I think there might be a worthwhile distinction to be made here. If so, we all need to be a bit clearer on it.

If qsort's comparison function is a callback, is that true of any function argument to any higher-order function? If so, why the additional term? If not, what makes a function passed as argument a "callback"?

When I read "callback", I interpret it as "function argument to be invoked to deliver a notification to a user-specified place". For example, to tell you that your transaction has completed, that your network connection was broken, that your write failed, that a log message is available, etc. Typically a callback returns no useful result and takes one argument, but sometimes takes no arguments, and in systems that don't have closures (like C) it typically takes an additional argument supplied along with the function pointer.

I think that's pretty reasonable.

Except that this is not a callback issue. It's an issue of circular dependencies - callbacks are merely the chosen dependency injection vehicle for this particular item.

The message pumping example the author gives has the exact same issue. The core here is that two items (database & logger) are mutually dependant. That will always be an issue. It doesn't matter if your promises are circularly dependant, your context objects, your callbacks, or your globals.

Static analysis won't help - fully resolving circular dependencies is equivalent to solving the halting problem, IIRC.

And so the only way to prevent those dependencies is to constrain the design in a way that prevents circular dependencies. That's where the ad-hoc rules mentioned in the article stem from. They artificially restrain the problem space to prevent (classes of) circular dependencies.

Yes, the core issue in the database logger example is that it's a circular dependency.

The point of the article is that callback-based APIs like this obscure the actual problems (like circular dependencies).

You're right that constrained designs prevents issues like these but callback-based APIs aren't inherently constrained. They allow anything to happen, which is why they conversely encourage errors like these (and are subject to pitfalls).

What would be an inherently constrained API, then? I've successfully managed to create circular dependencies in quite a few different styles, so I'd love to find one that saves me from myself ;)

Maybe inherently was the wrong word. Though you can imagine a logger API that was like this:

    typedef enum {
        CONSOLE_LOGGER,
        DATABASE_LOGGER,
        /* etc. */
    } LoggerType;

    void add_logger(LoggerType);

In this API you're constrained by what loggers you can add. It's not the totally unchecked free-for-all that a callback-based API provides. The user is strictly unable to shoot themselves in the foot.

But this limits the expressive power that callbacks provide. Sorry I don't have any other API recommendations. The only way I can think of to stay expressive while safeguarding against unintended abuse is to include code analysis.

I was thinking the same thing. It has nothing to do with callbacks.

> fully resolving circular dependencies is equivalent to solving the halting problem, IIRC.

I don't think so. Any good dependency injection framework would detect this kind of circular dependency.

Of logger and DB, certainly. The coupling introduced via locks is harder to detect. But most importantly, circular dependency between components might be OK in a path-dependent way - that's what I meant by fully resolving it.

Just analyzing the dependency graph of injected components is indeed just a topological sort.

Note that the queue doesn't fix the original database logging problem. The original form is still an infinite loop, the queue just makes it slightly less noticeable.

To make it noticeable, have the database's save_document() log a "saved element to database" message.

I find that promises are an elegant dependency resolution mechanism when requesting a resource asynchronously from disparate parts of an application. Promises still rely on callbacks, so the initial pitfall of calling `log()` in `get_database_handle()` could still happen. The main failure I see is that `log()` is called before the function has a chance to change the state of the application to reflect the fact that the resource is already being requested. Instead of creating a new variable to store the state of the request before logging, store a new promise, log, then resolve the promise.

> Time passes and your application becomes multi-threaded.

Is it realistic to think that any app starting with a bunch of assumptions about being single-threaded can ever just "become multi-threaded" without bloodshed?

Sure. E.g., AppKit on OS X is in general not thread-safe (still), and for many years apps were almost exclusively single-threaded. But apps and frameworks could occasionally spin up threads as long as they were guaranteed to do work in some bounded set of functions (i.e., as long as they were sure not to call into AppKit to update the UI...). So while the author's example is a bit contrived, I can see where it came from.

"Pure functions have no impact on any runtime state and that makes them always safe to invoke as a callback."

It also makes them UNIVERSALLY USELESS. Think about it: a function that definitionally has no side-effects is being called solely for its side-effects.

The article using that to wrap up makes it the equivalent of a complicated category theory proof that successfully proves that the empty set has TONS of amazing properties...

Pure callbacks aren't universally useless. For instance, the qsort() API can use pure synchronous callbacks effectively: http://linux.die.net/man/3/qsort

The article doesn't advocate always requiring pure callbacks. It's just offered as one possible way to make it easy to reason about the correctness of using an callback API.

Of course, the callback doesn't need to be pure. It just need to be side-effects free with respect to the caller.

Or presumably this is the same as a 'trivial solution' in physics (normally zero)? Makes the equations work, but generally represents nothing happening.

I don't see how you can ever pin problems with mutexes to callbacks. The author seems to confuse multi threading with callbacks. His example seems more like a mistake in recursion and improper structuring than something that went wrong because of callbacks. It's not the callbacks fault that you try to acquire the same lock twice.

Yeah problems with mutexes are ultimately the fault of the programmer.

The point that the article is trying to make is that because callback APIs don't expose their correctness requirements and because their correctness requirements are externally defined, they encourage programming error like this.

This article feels like it is talking about the pitfalls of not using OO principles (e.g. types, encapsulation).

I won't shed a tear if callback-hells are replaced by proper async APIs (like in Stream and Future-based dart:async, or the Thenable in the new JS standards), but this article seems to be misalinged.

Downvoters: care to elaborate?

I'll take a stab at elaborating.

The essay is about callback-based APIs, so types and encapsulation are out of scope from the start. It then expands the scope and observes that OO principles don't address the points covered. For example, it says that one of the "magic" requirements would be a way to state: "Do not add a callback that calls log() or acquires any locks held while log() is called." There is no type for that.

Other than pure functional code, there's no way to do that.

The essential equivalent for a multi-actor system is deadlock prevention. OO principles don't help there either.

You can get the problem with a stream. Consider a logger stream, where the listener opens a database connection to save the value then closes it, and the database adds 'open' and 'close' events to the logger stream. This will lead to an geometric explosion of events on the stream, because nothing at the API level says you shouldn't put those pieces together that way, other than the documentation.

I could go into the OO details of how this stream might be implemented in Dart, but really the OO nature of the stream API obscures the essential self-referential nature of the problem.

You can propose an equivalent counter-example if you want to demonstrate that those APIs really do solve the problem. I happen to agree with the well-written essay, and OO principles or "proper async APIs" solve nothing.

Promises are single assignment, so they solve the problem. As soon as they resolve, they're safe. Calls to them don't happen until they resolve. If you want to change the logger, you use streams, which will handle safe replacement for you (flatmap).

We do some crazy async HPC code, and during on-boarding, new members invariably get some initialization / error handling wrong with raw callbacks, and convert pretty quick to FRP after that. Unstructured async is crazy.

Yes, the example problem doesn't match to promises.

I don't understand the "change the logger" comment. The example was the logger sends a message to the listener, which opens a database handle, which sends a message to the logger, and repeat. There's nothing to changing. I don't see how FRP helps eliminate that cycle.

This might be what's happening, which would surface as no sensible declaration ordering:

//try making the logger first

var logger = require('myLogger')(backends);

var listener1 = require('mySync')(logger);

var backends = Rx.Observable.fromArray([listener1, listener2]);

//try making the backends first

var listener1 = require('mySync')(logger);

var backends = Rx.Observable.fromArray([listener1, listener2]);

var logger = require('myLogger')(backends);

From there, you'd switch to unsafe FRP methods (imperatively injecting into event streams, e.g., Subjects in Rx), which is the warning sign of weird cyclic behavior.

Further: the essence of OO is polymorphism, or late binding, which is to say that you're passing in objects to things that then call methods on those objects. That's a pattern of interaction that easily gives rise to all the problems the original article points out with "callbacks". In short, OO isn't a solution; it's a contributor to the problem.

> Consider a logger stream, where the listener opens a database connection to save the value then closes it, and the database adds 'open' and 'close' events to the logger stream.

Or just SSH to a server and run tcpdump without arguments :)

Reply on news.ycombinator.com