Understanding the Node.js Event Loop

I think its important, especially in posts like this that are targeted to beginners, that certain information be left out and not clarified (essentially making the masked parts "magic"). Otherwise, the reader is overwhelmed with information and really can't grasp the overall picture. I think this article succeeds on its goal of shedding some light on how the node event loop works at a high level, and its straightforward about its goals in that regard.

Imagine trying to explain to someone how math operations work in js, you wouldn't really want to explain how numbers are represented in the computer until you get to something that necessitates that understanding - like bit operations.

So, I'd just say it depends on your goals, if your goal is to explain precisely how every detail of something works, then yes I'd agree with you, there's no excuse for magic. But if you want to explain how a general concept works I'm fine with loads of "magic" holes you can fill in for yourself later.

The use of the term "magic" to describe spooky abstractions or odd behavior caused by referential opacity far, far predates Rails.

Right, I agree with this. There's nothing magic about any of this -- ultimately it's all just software, hardware and physics.

To describe something as magic in a serious context is toxic. We're not sleight of hand performers, we're engineers. It's more helpful to state something like, "this library is complicated and the internals are beyond the scope of this post". A pointer to documentation and source code on libuv would be even better. In face, this series of articles on libuv are a great start for those who are interested: https://nikhilm.github.io/uvbook/introduction.html

It's older than Rails, I remember the interpreter being described as magic in SICP. It was mentioned as such several times over the course of the book, and was somewhat thematic.

Until, of course, the final chapter (or was it the chapter before the final chapter), where you learned that magic isn't real.

Thank you. That was a fantastic talk. Short, to the point and useful :)

Case in point which tripped me up recently. Knowing that promise.then() callbacks aren't called synchronously[1], even if the promise is already resolved, I assumed that this:

    var tick = Promise.resolve();
    async function cpuBoundJob() {
      for (...many iterations...) {
        // crunch some bits
        await tick;
      }
    }

...would spread out a heavy calculation and allow other things to break in occasionally and do work. But based on tests, nothing ever got a chance to run until the whole job finished.

My current understanding (correct me if I'm wrong) is that promise.then() calls you back from a microtask, which means "await tick" doesn't teleport you into the next tick, just out of the current call stack into a new one at the end of the same tick, regardless of how many things are piled up waiting on the next tick. In other words the entire calculation remains one big CPU-blocking chunk. This modified version ended up working as expected, since setImmediate is (apparently) not a microtask.

    var tick = () => new Promise(res => setImmediate(res))
    async function cpuBoundJob() {
      for (...many iterations...) {
        // crunch some bits
        await tick();
      }
    }

[1] https://promisesaplus.com/#point-34

btw, the params order for callbacks is not very node-y in HyperDex. It should be (err, result) instead of (result, err).

Putting the error first would enable Async or Promise libs to work...

Node's event loop is inherently cooperative multitasking. It's not trivial to simulate preemptive multitasking without writing/compiling your code to giant state machines. There are different use cases for the two, especially since any CPU-bound task performed in the event loop will cause stuttering in your app.

In a node process you can't run 2 things at the same time without forking an external process. That's why these libraries exist. A complex synchronous computation will block the only thread available in nodejs. Sure you can spawn workers, but your node workers will have the exact same issue.

> all reinvent the concept of threading poorly

because "user scripts" in node have no concept of threading to begin with. There is no concurrency in a node process, only non blocking I/O.