Hacker News Reader

Comment by Matthias247 | Hacker News Reader

Channels like the one in the link are pretty easy to create. However that's more like Javas BlockingQueue than like Go channels. The power of Gos channels is in "select", which enables a lot more sophisticated synchronization mechansism then just getting data from thread A to B. E.g. waiting for events from multiple sources or cancellation support.

Building channels with select functionality is a lot harder.

An interesting thing is the equivalent of pthreads on windows (WinAPI synchronization primitives) actually provides something which has some similarity with select out of the box: WaitForMultipleEvents, which is also quite powerful. However that is also a still a lot harder to work with than Go's channels, since there are additional sources for errors like HANDLE invalidation, which you can't have in Go where channels are garbage collected references.

View on HN

What is even more fundamental and useful than select is non-blocking reads (which the implementation I posted supports), once you have that you can write your own select. It's not magic, select has to traverse the list of channels just like any other code. The only reason it feels like magic in Go is that it's the only way to read a channel without blocking.

Select is a kernel level interface. I don't know how it's implemented at the kernel, but there is no technical reason why it can't follow from an interruption to an array index and a file descriptor without ever passing through a list.

Besides, how do you block in user space when there is nothing to read?

timerfd?

Are you saying they are allocating a separate file-descriptor for each channel just to be able to select? I guess you could do that if you wanted to but it's far from the first thing I would try. I remember reading an article on golang about select and having to traverse the list, but I don't really have the motivation to dig deeper right now.

Looping with unblocking reads is plenty fast enough though.

As for waiting on any event without select, adding an optional condition-variable pointer to each channel that is pinged when data arrives is the easiest approach I can think of.

Well, select is well known for not scaling well so AFAIK it may very well use a list. What I'm saying is that kernel code can be much more performant than anything you may create on userland because you have access to the interruptions. And that code would be "magic" in the sense that you really can't replace it with a library.

In fact pinging condition-variables after a channel successfully is read is a much more expensive emulation of that same behavior. I'm curious if the Linux kernel actually exports something like that.

The non-blocking read parts is also easy, but that's also only one use-case for Go's select. The harder part is the generic "wait until one of the select cases can be taken".

You can't implement this with non-blocking reads alone. If you would, it would be something like busy spinning and iterating through all select cases and trying none-blocking reads until one succeeds. But that's just not efficient.

You need a mechanism instead which also registers a pending select at each event source and unregisters it when done. And the event source (e.g. channel) must wake up one or more select when it's state changes.

Adding a condition-variable per select and keeping a list of pointers in each channel that are pinged when data arrives is the easiest approach I can think of. Haven't tried that one though, I have honestly never run into a use case where I needed select for channels outside of Go.

Reply on news.ycombinator.com