Timeouts and cancellation for humans (2018)
vorpus.org
vorpus.org
I think it's unfortunate that even new languages still treat timeouts and cancellation as an afterthought. For example, every Go program I've ever written says:
select {
case <-ctx.Done():
return nil, ctx.Err()
case thing := <-thingICareAboutCh:
return thing, nil
}
return <-thingICareAboutCh, nil
(Getting off topic, this is why I avoid mutexes and other concurrency operations that aren't channels; you can't cancel your wait on them. Not being able to cancel something means that if there are any bugs in your program, you'll find out when the program leaks thousands of goroutines that are stuck waiting for something that will never happen and runs out of memory. Even if the thing they're waiting for does happen, the browser that's going to tell someone about it has long been closed, and so you'll just die with a write error when you finally generate a response. If you have a timeout and a cancellation on every blocking task, your program gives up when the user gives up, and will run unattended for a lot longer.)
Mutexes are designed to solve different problems -- albeit with overlap. Channels are great if you have a one to one relationship or even a many to one but they're not so good with many to many relationships. That is where you need several routines to have read access and several routines to have write. I'm not saying they can't be used in that way but if you're not careful it's even easier to have goroutines "stuck waiting for something that will never happen" with channels than it is with mutexes. So I'd be very nervous about shoehorning channels into a design they're not well suited for.
Also you can cancel your wait on mutex if there is another goroutine which unlocks that mutex. In fact if you browse the Go source you'd see that channels use mutexes themselves.
So does parking_lot: https://docs.rs/lock_api/0.3.2/lock_api/struct.Mutex.html#me...
We’re used to lexical scope: it’s easy to reason about, and it is a really good default. But sometimes it makes sense for one function to apply settings for all the functions it calls, without interfering with other functions, scopes, threads or processes (like setting a global would).
It’d be nice to be able to say ‘this function should timeout within 10 ms’ and then any function called will just automatically timeout.
Go’s contexts integrate timeouts and cancellation, and permit one to add any value, should one wish to, but you have to be disciplined and add a context argument to every single function. It’d be better, I think, to support it natively in the language. Lisp does this: any variable declared with DEFPARAMETER or DEFVAR is dynamic, and you can locally declare something dynamic too.
One can fake dynamic scoping with thread-local storage and stacks or linked lists, if one needs it, but it can get ugly.
Dynamic scoping doesn’t get the attention or respect I think it deserves. It’s arguably the wrong thing by default, but when it’s useful, it’s really useful.
[1] https://vorpus.org/blog/timeouts-and-cancellation-for-humans...
See ayo for a very rougth draft of the idea: https://github.com/Tygs/ayo
Stackless Python has this ability - it can even resume suspended functions: https://stackless.readthedocs.io/en/latest/library/stackless...
Additionally, suspended tasklets can be serialized, then saved to a file or sent over the network and resumed elsewhere!
The two big issues where: 1) "User managed" namespaces 2) Multi-process programs, or perhaps multi scope programs
The issue with "user-managed" namespaces is basically one of lexical scoping. In traditional dynamic scope, all variables in the dynamic scope are global. As such people tend to accidentally overwrite other people's variables. I've thought through some ways around this, but none have been elegant.
The more important, and foundational one takes some explaining. Consider some code:
type Work struct {
Ctx context.Context
Result chan<- int
}
func NewWork(ctx context.Context, c chan<- int) *Work {
return &Work{
Ctx: ctx,
Result: c,
}
}
type Worker struct {
c <-chan *Work
}
func NewWorker(c <-chan *Work) {
return &Worker{
c: c,
}
}
func (w *Worker) Work(ctx context.Context) {
for work := range w.c {
// We now have two contexts,
// the context of the work call, and
// our "local" context"
// Both are usefull!
}
}
func main() {
workers := make([]*Worker, 0, 10)
workChan := make(chan *Work, len(workers))
for i, _ := range workers {
workerContext := context.Default()
workers[i] = NewWorker(workChan)
go w.Work(workerContext)
}
http.HandleFunc("/bar", func(w http.ResponseWriter, r *http.Request) {
httpContext := context.Default()
work := NewWork(ctx, result)
fmt.Fprintf(w, "Got Result, %q", <-result)
})
log.Fatal(http.ListenAndServe(":8080", nil))
}
I wonder if there's a reasonable way to apply this approach to more general-purpose code.
Since gevent is generally used by monkey-patching all standard IO, most code doesn't even need to be aware of this feature - it just treats a timeout as an unhandled exception.
From the user's perspective, it can be used simply as a context manager that cancels the timeout on exit from the block:
with gevent.Timeout(10):
requests.get(...)
response = None
with gevent.Timeout(10, False):
response = requests.get(...)
if response is None:
# handle timeoutThere is a reason why most progress indicators suck, and it's because it is in general surprisingly hard to write one.
All single-threaded epoll-or-equivalent-based network servers hit this problem sooner or later and become multi-threaded network servers (also, there is async DNS resolution, but that's a whole separate barrel of worms).
That's kind of the point. Programs shouldn't assume that any particular file is local. Any file may be hosted on a network mount, and that means network-style asynchronous interfaces should be used to access the data.
Syntactically, I think it is worth distinguishing between things that can time out and things that can't, because you usually need to do some sort of cleanup on timeout.
In fact as far as I can tell, the cancel scopes provided by Trio with the async await syntax are exactly isomorphic to Scala's tasks from the cats library (where they are called IO).
Also I'm not sure I understand the author's preference for thinking of timeouts as level-triggered rather than edge-triggered. While it's an interesting way of thinking about the problem, and would be the natural way a timeout is implemented in an e.g. FRP system (a lot of flavors of FRP are essentially entirely level-triggered systems), it doesn't seem like the way you'd implement things in a non-reactive system. What's wrong with just killing the entire tree of operations (as is usual when you propagate say, an exception) on a timeout, or from a top-down manner when you put a timeout on a task?
Timeouts are fundamentally tied with concurrency (they are a concurrent operation: you're racing the clock against your code and seeing who wins) and to me the tricky thing about timeouts is exactly the same trickiness that you face with concurrency, namely shielding critical sections. How you decide to pass timeout arguments seems like a secondary concern. Just like with normal concurrency, you need to make sure that certain critical sections are atomic with respect to timing out, either by disallowing timeouts during that critical section (you therefore need to make the critical section as small as possible, ideally a single atomic swap operation) or implementing a reasonable form of rollback. (Of course you can always take the poll-based approach where you poll for timeout status, but again this is just a specialization of a general concurrency strategy)
Here's one example of such a library, though without a bit of FP background it probably doesn't make a great deal of sense:
https://typelevel.org/cats-effect/typeclasses/concurrent.htm...
On a sane system, every little thing must be cancellable, otherwise nothing really is. This interface fails because of this.
Maybe the developers collected lots of data and said p99 latency is X, so let's set the timeout X+alpha. If something takes drastically longer than most requests it's probably something going wrong. Maybe your latency was the 99.99999 and the timed you out.
Or yeah the devs just guessed and guessed badly and the timeouts cause more problems than they solve.