I feel like this glossed over how when entering something like epoll, Go can optimize switching to another goroutine. Wouldn't the epoll block until the calling kernel thread had a file descriptor ready?
IMO this partially undermines the "Go doesn't have colored functions" claim. 1k goroutines calling read is fine, because read is multiplexed, but 1k goroutines calling stat will be painful, because you get 1k kernel threads. Go's "red functions" are those which require a dedicated kernel thread.
In principle, on linux, you should be able to multiplex stat anyway via io_uring.
It sounds like blocking calls switch to a system stack and return the Go stack to the executor pool, but I don’t have source links to back up that claim.
You wouldn't necessarily need to make use of this feature to implement something like Go's scheduler. Jumping to the semantics of epoll skips most of the interesting bits about how Goroutines work. The source code will always beat a blog post. If you're curious, you could work backwards from here: https://go.dev/src/runtime/netpoll_epoll.go Note that interface does indeed expose the timeout argument to the higher-level scheduling code; see line 92.
However if memory serves, these calls are reasonably expensive (on the order of microseconds, and _worse_ if readiness requires putting the current thread to sleep and then waking it up again). However https://www.ardanlabs.com/blog/2018/08/scheduling-in-go-part... says goroutines are closer to 200ns.
However the same source above does a decent job of explaining how epoll is being called "out of band". I am guessing when they talk about context switching costs, they are doing some sort of amortization of these system call costs. Since the actual OS threads try to remain CPU-affine (thread-per-core, LMAX disruptor etc. etc.), a lot of the OS scheduling costs can be skipped.
Thank you for the pointer to the source code. It gives me a starting point.
Are you saying that the system call is performed on another kernel thread while goroutine continue running on the GOMAXPROCS threads?
You can think of an event loop multiplexing across multiple sources of events: for example a local ready queue, a remote queue (for tasks coming from other threads), an io-queue, a timer queue. Epoll is used for the io-queue. If the local ready queue is empty (i.e. there are no runnable coroutines), the event loop will block in the epoll call, possibly until the next timer expiration if any; remote queue operations will wake-up the poll via somethink like event-fd. If the local queue is non-empty, epoll is called with a 0 timeout every N local queue pops (actual scheduling logic and priority is application specific of course).
Polling epoll with a 0 timeout is a bit wasteful, for high performance applications you can use io_uring so checking for non-empty queue is very cheap and/or use some network bypass library that makes epoll cheap. There is also old-school readiness notification via signals but I doubt that still see any use.
As you suggest, you could also have a dedicated thread per event loop just running poll (for example a sibling hyperthread), but it seems wasteful.