Comment by majke | Hacker News Reader

Comment by majke | Hacker News Reader

Ok, I'll chime in here since all the comments are missing the point.

If you are excited by golang-style in C, then Mr Sustrik's libmill is for you http://libmill.org/tutorial.html

LibDill is a thin experiment "above" of libmill. The main problem it tries to solve is "cancellations". For the uninitiated, in normal programming languages there is no way to cancel/interrupt a tread/coroutine from outside. Look at the mess in pthread_cancel(3) [1]. In golang as well, there is no way to cancel a coroutine.

Why this is important? Well, read Mr Sustrik's notes [2]. But basically - imagine you want to define the time limit for a completion of some task. Or - imagine what should happen if a HTTP request comes, initiates some actions (SQL quieries?) and then disconnects. Normal programming languages have NO way of expressing that - "client went away, stop processing its sql queries".

Libdill is an attempt to solve this. I must admit, I'm not fully on board with the "structured concurrency" train of thought [3] , but the whole prospect of defining semantics of killing/cancelling coroutines is absolutely amazing.

Also, review the golang "context" mess, which AFAIU tries to work around the same problem. [4]

[1] http://man7.org/linux/man-pages/man3/pthread_cancel.3.html [2] http://250bpm.com/blog:71 [3] http://libdill.org/structured-concurrency.html [4] https://golang.org/pkg/context

Author here. Thanks for the nice summary. You've expressed it better then I would.

I am not 100% sure about the structured concurrency concept myself, but comparing it to structured programming feels reassuring. Basically, a tree structure (whether a syntactic tree or the tree of running coroutines) is probably the most complex way to structuring stuff that's still tractable by a human being. Once you go beyond that, you'll eventually get lost. Thus, tools enforcing a tree-like structure are likely to be helpful to keep the program maintainable.

I like the idea of structured tree-like concurrency, fits my brains.

About your API though.

Why is there a load of inet/ip/socket functions, shouldn't they be in a separate lib? Why is the opposite of the fn int go(expr) the fn int hclose(int) and not the fn int stop(int) or something like it. open/close – go/stop – and so on …

The socket functions were once in a separate library but it's just easier to use like this. Linking with 1 library, not having to care about version mismatches etc.

My recommendation would be to just not use the socket functions and link with the library stitically. That way the unused code will be discarded.

As for hclose() that was just mimicking the POSIX close() function.

Gotcha. Okay.

We use D fibers. When we want to cancel a fiber we throw an exception at it. When it next wakes up, it'll throw (much like sending an exception to a Python generator) -- which will make sure RAII cleanups take place and are not missed.

Getting that right at the implementation level is quite hard; it also requires support at the OS level that isn't always there (e.g. interrupting a blocking system call).

Then avoid blocking system calls.

When you can't, defer them to a thread that can be ignored.

That's a huge footgun in an environment where blocking calls are the default way to get almost anything done, and some calls cannot be made non-blocking.

True, but we don't have much trouble with it though. We don't directly call syscalls but use our own wrappers that take care of this for us.

Sorry for the late reply, but I just saw this reply. Can you share who "we" are? I'm always interested in new practical systems for concurrency.

Not sure why don't you like go's context model. It works pretty well in our production system. I can see people would like to pass the context implicitly. But I don't have a strong opinion. Explicitness is not that bad.

Go context is a step in the very right direction, but we have not yet the destination. First there is legacy libraries that do not implement it. Second even system-level support misses pieces. For an example, try to cancel a read from a pile using it.

Go's context is fine to signal cancellation, but it lacks a way to wait for that cancellation to complete gracefully. And the fact that it is also an arbitrary bag of values is concerning. It worries me that it started "polluting" the stdlib's API without solving those issues first. Dave Cheney wrote a good article about it: https://dave.cheney.net/2017/08/20/context-isnt-for-cancella...

Context is problematic because it infects everything. If a function gains context support, every call to it has to be updated, and if you're going to do it right (and not just use context.Background()), every function calling those functions need to be updated, and so on. And the context has to be passed downwards, too. It's turtles all the way up and down.

There's been talk of adding context to io.Reader and io.Writer, which would of course affect pretty much every single Go codebase. For backwards compatibility/convenience you therefore see a lot of APIs with complementary functions Foo and FooCtx, one which takes a context and one that doesn't. That's awkward, and even less elegant when interfaces are involved.

And the network packages (sockets, etc.), of course, have a parallel, non-complementary mechanism to provide timeouts and cancelation that doesn't use contexts. (The key/value pair context stuff is also unfortunate, in my opinion. It's not typesafe, and results in awkward APIs.)

All of this happened because goroutines, as designed, cannot be forcibly terminated. I don't know what the better solution is. I'm not sure why the runtime cannot provoke a panic at the next context switch, but I bet there's a good technical reason. It's unfortunate.

But it's not unreasonable that a large part of any codebase needs "contexts", and so if everyone wants it, why not make it mandatory? Make it an implicit argument threaded through every function call, a bit like "this" in languages such as C++ and Java. The compiler could optimize it away from those codepaths that are guaranteed not to need it. Go has a lot of built-in generic magic, adding some global helpers to set a scope's context wouldn't be too bad.

The other route is to imitate Erlang's processes, which have their own heap, and so killing a process doesn't leave cruft behind. Given how goroutines are allowed to share memory, that's not likely to happen.

I see. Now I see a strong point of favoring implicitly passing the context - attaching the context to a go routine local variable (no such thing yet), so the aspect of context passing can be decoupled. Of course for some logic we probably still want to do some explicit context stitching, but that won't pollute most of interfaces. Would that address your concern?

Reply on news.ycombinator.com