Hacker News Reader

Comment by galaxyLogic | Hacker News Reader

One thing to note is that async programming like in JavaScript is "threadless" programming which means all your data-structures are by definition thread-safe.

It's one less complication in your logic when you don't have to think "what if some other thread modifies this data while I am doing it?". And you never get hard-to-detect and hard-to-fix errors caused by that nor do you need to try to set up locks that might cause performance problems.

So some might say that threads and locks and synchronization primitives are the solution while others might say they are the problem.

View on HN

Note though, that many of the problems that come up with threads can still come up with single-threaded async, especially when building a server. As soon as you await, any object visible to another promise chain can potentially change out from under you, and this can cause analogous problems to multithreading. One example where this can get hairy is if you have a database connection shared between two promise chains running "in parallel". You'll get undefined query ordering, which can cause very bizarre behavior.

Don't get me wrong: it's still easier. But there's an interesting property I've noticed of statements of the form "by definition, problem X is impossible when you have Y", which is that X-analogous problems absolutely can happen, and they are often more complicated to solve. For another example: "by definition, memory leaks are impossible when you have a garbage collector".

I once had to write a lock for JavaScript, it was beautifully simple.

You're making me curious enough to want to see it. Would it fit in a post here or on a short gist?

It can just be a boolean variable called locked?

Almost. In threaded programming a thread will wait/block until the lock is released and then make use of the protected resource. In async world you would probably just check some flag and if it's "locked" then register a callback to some "unlocked" event to mimic this.

"Promises" are a lot like locks.

Lookup twisted deferred locks. https://twistedmatrix.com/documents/9.0.0/api/twisted.intern...

Looks nice. How is this implemented?

Not gp, but I used Twisted Matrix for a few years. It is usually implemented with a boolean flag and a list of client waiting to be called back when the lock is released.

So users of the lock would call acquire and get back a deferred (a promise in other language), but say in the most basic way, a callback is passed to it. When lock can be acquired, that callback is called (or deferred is fired).

When done with the resource, must call the release. Release then will trigger the lock to inspect the list of available waiting clients, pick one and fire their callback letting them know they can access the resource, and so on.

https://github.com/twisted/twisted/blob/trunk/src/twisted/in...

Yep.

  let lock;
  
  const log = (msg) => {
    lock = new Promise(async (resolve) => {
      await lock;
      console.log(msg);
      setTimeout(resolve, 1000);
    });
  };

  'hello world'.split('').forEach(log);

I don't have access to the code anymore, unfortunately. But it kept track of a list of people waiting on the lock and then notified the first caller off the list when the lock was released. It was only beautifully simple with respect to what it would have been in a non single-threaded evnironment.

I needed it in order to use a library that had some annoying global state in it.

Not sure if this helps but I hacked this together the other day to avoid adding a real queue to a very simple application. It uses event queue ordering semantics to get FIFO behavior.

The read/write combinations to the semaphore variable need not be atomic since we know that if this thread is running there are no other threads that can be reading and then modifying it.

    MyObject.queueSemaphore = N;
    MyObject.emitter = new EventEmitter();

    MyObject.queue = function( ... ){
      // if this run can take place go
      // otherwise wait for a run-complete to try again
      if(this.queueSemaphore > 0) {
        this.queueSemaphore--;
      } else {
        // must return a promise that resolves and possibly requeues
        return new promise((resolve, reject) => {
          this.emitter.once("run-complete", () => {
            resolve(true);
          });
        }).then(() => this.queue( ... ));
      }

      this.promiseBasedProcess()
        // ... work
        .finally(() => {
          this.queueSemaphore++;
          this.emitter.emit("run-complete");
        });
    }

Here's a lock implementation I made just barely as a proof of concept:

https://bl.ocks.org/johnsonjo4531/99256568deaf8c0f1685793a4a...

Basically it is used like this:

```

  var lock = new AsyncLock();

  (async () => {
    for await(var _ of lock) {
      console.log("here 1a");
      await sleep(1000);
      console.log("here 1b")
    }
  })();

  (async () => {
    for await(var _ of lock) {
      console.log("here 2a");
      await sleep(1000);
      console.log("here 2b")
    }
  })();

  /*
  When Lock is working should print:
  here 1a
  here 1b
  here 2a
  here 2b
  Without the lock it would print:
  here 1a
  here 2a
  here 1b
  here 2b 
  */

```

If you want to check out the implementation go to the blocks link. (which is basically just a gist that you can run). Here's the actual gist link: https://gist.github.com/johnsonjo4531/99256568deaf8c0f168579....

It requires Chrome 63 (at this point Chrome Canary) because it uses async iterators. I used this because I can close the lock automatically in case an error is thrown, the loop is broken. It only iterates one value and only after the lock is unlocked.

Concurrency !== parallelism. You can have concurrency, as in async javascript, and all the issues that come with concurrency, even when no two execution paths are executing simultaneously. The main benefit is that your lock structures don't have to worry about being accessed in parallel.

Valid point that async is easier than parallel. I've done a lot of both, and parallel is harder. But, I realize that some people say that concurrency in software is not parallel, but I think that's a poor choice of terminology that is prone to confusion and argument. The word concurrent is literally synonymous with "simultaneous". We have a good word for asynchronous. Let's use async instead of saying that concurrent means async but not parallel in spite of it's common meaning.

> Let's use async instead of saying that concurrent means async but not parallel in spite of it's common meaning.

No, let's not. They mean different things. The execution of Python generators is concurrent with your code, but not parallel or asynchronous. And it's also completely possible to have concurrent synchronous parallel execution [1].

[1] https://en.wikipedia.org/wiki/Bulk_synchronous_parallel#The_...

I think you agree with me- the comment I replied to appears to conflate concurrency and asynchrony. The article is about async. I’m saying use async when talking about async.

And I avoid using concurrent if I can. It does not have a strict CS definition, and it is prone to miscommunication. Many people use “concurrent” when they are talking about parallelism. Some people’s CS definition of concurrency is not very well matched with the common non-CS definition of the word.

The dictionary definition of “concurrent”, according to Google, is: “existing, happening, or done at the same time.”

synonyms: simultaneous, coincident, contemporaneous, parallel

Using “concurrent” to mean not parallel, or even out of order or not necessarily parallel, just seems like a bad idea.

> I think you agree with me- the comment I replied to appears to conflate concurrency and asynchrony. The article is about async. I’m saying use async when talking about async.

I'm not sure we do. The comment you replies to does not conflate those. "You can have concurrency as in async Javascript" is exactly correct: an example of concurrency is async in Javascript. People don't say "an example of asynchronicity" because that's not what they want to talk about. You can't really change the word unless you change the topic they are trying to talk about.

> And I avoid using concurrent if I can. It does not have a strict CS definition

It very much does. It means an operation overlapping in time with another one, i.e. one beginning before the other one ends. The colloquial definition is irrelevant in a technical context. It's like suggesting scientists stop using the word "theory" on the grounds that laymen don't use it properly. No, people should just use the right word in the right context.

To deal with this exact issue, I wrote a 'Promise Queue' for use in our codebase. The implementation of this queue is simple. It is initialized with a resolved promise. Each operation (i.e. a function that returns a promise) is enqueued by calling `.then(theFunction)` on the queue, while replacing it with the returned value from the `.then(...)` invocation. This ensures that two different operations do not interleave.

You lose some potential speed this way, but you are able to reason about the states your program will take far more easily.

That sounds sort of similar to what Selenium webdriver does with promises, although it does it so that the promises can be automatically managed without chaining .then calls. In webdriver's case, this causes async/await to deadlock. Does your queue system have that problem, or is it something you had to work around?

I ask because at work, we have a lot of browser tests that we'd like to start transitioning off of that deprecated queuing mechanism. If I can rip out Selenium's version and provide my own mechanism that doesn't deadlock on async/await, it would make the transition a lot easier.

With asynchrony you get out of order execution and non-determinism by definition.

That you don't have the concerns of shared memory concurrency, that's actually not such a big relief, especially given the right abstractions. And the underlying problem, that you need to deal with concurrency, doesn't go away.

I've just written an article describing how JavaScript's Promise leaks memory in "then" chains and that it shouldn't: https://alexn.org/blog/2017/10/11/javascript-promise-leaks-m...

The reception was actually negative, having lots of people telling me that I'm concerned about a niche use-case, but functional programming isn't a niche at all, being one of the two known ways to write reasonable, composable code that deals with concurrency and asynchrony, the other way being to have a draconian borrow checker, like that of Rust.

Mainstream developers are in general deluding themselves that the tools they are using are adequate, in spite of repeated evidence to the contrary, whereas most current breakthroughs in productivity were born out of functional programming, starting from garbage collection, but also actual libraries that people are now fond of, like React.

Haskell does shared memory concurrency. The JVM / Scala does that too. In Haskell you work with things like IO, MVar, STM (transactional memory) and many more. High level abstractions that have been borrowed by Scala as well, my current languages of choice.

I feel much more confident writing concurrent Haskell or Scala code, with 0% test coverage, then I am writing JavaScript code with 100% code coverage.

> all your data-structures are by definition thread-safe.

It is thread safe from data being accessed by two different threads, well because there is only one thread running. But surprisingly it is not free from application level data races in general, and you might still need async mutexes or lock type constructs.

That is because it is possible to have multiple callback chains each partially modifying the same data structure, or trying to write to the same tmp file, or modify some global resource.

In general with callback programming each callback chain specifies a sort of a poor man's IO concurrent context of execution. In a thread you spawn a thread to execute some code in a function, so it executes instructions A, B, then C. And there can be another thread executing the same A, B, C instructions in parallel. With callbacks it's the same idea, a chain of A, B, C callbacks can execute IO requests in parallel. Two client request coming almost at the same time, would run through A, then B, then C functions. A, B, might be called by first request. Before it finishes, and calls C, another request might start A, B chain again.

IMO thread programming via locks, mutexes and such is in the same category as assembly programming - they're necessary low level primitives that are extremely easy to get wrong, and hence they should ideally be left to specialized experts, who can use them to develop higher level abstractions (e.g. worker pools, event queues, tasks) that normal application developers can use for most of their use cases.

Agreed. I've been programming primarily in Scala the last few years, which has great high-level concurrency abstractions, and the code honestly stays readable and easy to reason about. We get the odd concurrency-related bugs, but they're reasonably rare, no more common than people screwing up things like time, text encoding, not properly using db indexes, etc.

Thread-based concurrency need not be overly scary. Any concurrency model will take some time to wrap your head around, but with the good ones, once you get comfortable with them, they should feel very safe, intuitive and deterministic. Scala's approach of "Futures + immutability when you don't need mutable state, or Actors when you do need mutable state" has worked well for every problem I've encountered in the last few years, and I've certainly heard great thing's about Go's approach too.

ES6 has improved the state of concurrency in JS, it's now similarly easy to read and reason about as Scala's model, but the lack of actual parallelism is a pretty big limitation - often a deal breaker for server-side code, and still often an issue client-side.

> but lack of actual parallelism is a pretty big limitation - often a deal breaker for...

Well, in my experience, it has been a deal breaker in a select few, rare cases. But often?

Do you routinely come across problems that could NOT be solved using a multiprocess approach (in Node) or webworkers on the client side? Could you share some of these scenarios?

All the time. Web development is a sad excuse for experience in HPC environments. But node guys always think I/O processing in an event loop must work for everything!!!!

With that approach we can go back to the happy openmpi/mpich days and take the gargantuan performance hit that ended MPI clustering for any serious real time scientific processing.

> All the time.

Again, I must press for examples/real scenarios.

30 arc second permutation on a TC forecast track as it impacts multiple points of interest across multiple impact models. Heavily parallel amenable and in RT not an async endeavor.

Not giving you more 'hints'. You webdevs are kind of dull anyway. I/O this and I/O that and you'll accept untrusted input until the sec industry all drive ferraris on your mistakes.

How about database access?

Even node uses a thread pool under the covers for that.

I like the syntax that C# provides to make the code easier to reason about.

I agree, and things like the TPL make some uses of parallelism remarkably simple

Disagree. Threaded programming is not that difficult. Design matters. If you are trying to back fit threads into legacy code that has a traditional process based model with massively shared data structures it can be a little ugly.

Most peoples problems come with being over clever with their per thread data/stack, priorities, affinities, attributes, etc..a simple threaded program with a couple of locks for shared data and a flat schedule is efficient and easy to understand on multi-core systems.

Event loops and aio are nice but they are not a superior standalone solution as evidenced by nodes use of libuv.

Stupid question. Does this mean that a web server written in node is running single-threaded? I know that there are callbacks and promises to hand over control between different execution flows. But doesn't running on a single thread put an upper limit on the amount of work a server can handle?

And, if the solution is to spin up more servers with access to the same database, doesn't that mean that we are now having multiple threads accessing the database concurrently? Much like, say, Python Django?

> Does this mean that a web server written in node is running single-threaded?

Yes.

> But doesn't running on a single thread put an upper limit on the amount of work a server can handle?

Is not about how much work it can handle, it's about how much it can offload. Async servers can handle a much greater volume of I/O bounded tasks. So it can handle more connections. When the task is CPU bounded you can either create a thread (which does not really scale well) or offload it to some other servers that can scale horizontally (i.e doing micro services)

> And, if the solution is to spin up more servers with access to the same database, doesn't that mean that we are now having multiple threads accessing the database concurrently? Much like, say, Python Django?

Yes, to take advantage of all CPU cores you have to create more server instances. But why would they talk to the same database instance? It could be a replica or a shard. You can even have a pool of shards connections per server instance.

The problem with offloading to a thread is that you can't structurally share data with another thread. So any copying of e.g. arguments can be very expensive, and awkward/inconvenient as well.

I agree. But like always, trade-offs.

> The problem with offloading to a thread is that you can't structurally share data with another thread.

That's really a JS issue, not a general async programming issue, though.

If the other thread/process is running on a different machine (or you want to keep that option open) that's what you're going to have to do anyway though.

Yes, but that's a big "if".

Sometimes you want to use a thread to keep the CPU from blocking (e.g. long database operations).

Also, sometimes you want to use the CPU to its full capabilities.

Database access, likely the most common I/O operation from a web server, is certainly running in another process (or on another machine) - I'm not sure that "if" is as big as you suggest.

>Sometimes you want to use a thread to keep the CPU from blocking (e.g. long database operations).

On the contrary. That's the prototypical use case for non-blocking event based IO. No threads needed.

>Also, sometimes you want to use the CPU to its full capabilities.

You can use all CPUs by using multiple processes. That's not an issue.

Threads are useful when you want to run multiple algorithms in parallel on the same bigish in-memory data structure, especially one that has a lot of pointers.

Something like an in-memory graph database or a desktop application that lets you work with huge files in-memory, or even complex user interfaces. For instance, I'm not convinced that the cross process bridging that React Native has to do is a great idea.

So yes there are use cases where threads are very beneficial. But on the server side it's essentially the database/analytics systems themselves, not the code that accesses them.

> > Does this mean that a web server written in node is running single-threaded?

> Yes.

Just to expand, this (like most things) is a simplification, as I'm sure nitely is just being too brief to explain. It's true for Hello World, and a little further, but real-world web servers in non-trivial contexts typically utilise techniques like clustering, workers, and other ways of delegating tasks to external processes.

>Yes, to take advantage of all CPU cores you have to create more server instances. But why would they talk to the same database instance?

Why not? Unless you've exceeded the capacity of a single DB and have a real use for sharding etc, it would not make sense to have difference DBs (+ replication overhead) for different Node processes.

The parent is asking how comes async I/O can handle higher volumes of requests given it's single threaded and even if it does how comes the database is still not the bottleneck. I answered both of these questions.

Yes, and I had an objection with the answer to the second question, that it might leave the impression to the parent that sharding or replication and pooling is required to have good DB IO performance with multiple Node processes -- when in practice it might or might not be an issue.

You can have a 12-processes node cluster and still not need a second db.

I'm a noob to concurrency, but my current understanding is that a "thread" is "virtual". Fundamentally, your CPU has finite cores so no matter how many threads you spawn your (individual) CPU cores will resolve the threads into a synchronous set of instructions to execute.

Having a single threaded (asynchronous) server simply displaces the abstraction of threads.

Please feel free to correct if I'm wrong about this...

Yes, but you need at least as many thread or processes as you have cores to take advantage of them.

A multicore cpu is wasted on single threaded program unless you are lucky enough that your problem is so embarassingly parallel that can be handled by N indipendent the processes

No, async & single-threaded is not an abstract replacement for threads. Single-threaded async code can only use 1 CPU core at a time. For example, JS running in the browser without web workers. Multi-threaded uses multiple cores simultaneously.

Whether a thread is "virtual" - I don't know what that means to you, but threads are a primitive the OS provides. Some CPUs have hardware support for threads, and some don't. So, I suspect they are less virtual than you're thinking.

> Does this mean that a web server written in node is running single-threaded?

JavaScript is single threaded. Node.js is not.

Node executes the entire JavaScript code in a single thread. However, the I/O requests dispatched by the JavaScript code (file I/O, network I/O, db I/O) can be executed in separate threads.

That is why Node is efficient for applications with lot of I/O (typical web apps), and can still provide a default thread safety for global objects.

But I think, it will not work efficiently for number-crunching CPU intensive applications.

Node is written using libev, which under the hood uses system calls like select, poll, epoll etc (whatever is fastest for the combination of the IO task at hand and the current kernel) to provide an abstraction called an event loop.

It acts as an intermediary between your application code and the kernel, notifying you as soon as some IO action was completed by the kernel via an event callback.

This notification is provided to you as a single queue of events; the event loop is hence single threaded.

The important thing this facilitates is making it easier to reason about your application code, since you can be assured that only one of hundreds of callbacks in your application can be running at any one time.

Does it put an upper limit on the amount of work a single server process can handle? Depending on your use case, possibly yes. NodeJS shines when most of the work each call to your server involves mostly IO, i.e. are IO bound tasks.

If on the other hand, if any of the calls are CPU bound (some complex mathematical calculations say), you're probably going to hit this limit much sooner.

Even in cases where you have to run CPU bound tasks, it is far 'simpler' to offload these to an entirely different process that uses some sort of IPC to run the calculations and communicate the completed results back to your main server process, rather than spinning up a new thread in your server process to handle those CPU-bound tasks.

Does it now not possibly involve multiple threads accessing the database concurrently? Well, yes. Databases though are rather good at handling races. Most mainstream databases provide some sort of locking mechanism to make sure that some shared record cannot be erroneously modified by two processes at the same time.

If database locks are not for you, there are other solutions possible for these kinds of issues as well. By implementing a proper message queue, you can filter out calls that access this shared record into a separate synchronous queue, while all the other calls can be made to the DB simultaneously.

Why bother with mutex locks and races in your application code when other people (authors of libev/databases) are willing to do it for you?

>Does this mean that a web server written in node is running single-threaded?

Yes -- node is by default a single threaded, single process server.

>But doesn't running on a single thread put an upper limit on the amount of work a server can handle?

Not any more than this is the case with Python, PHP, Rails, etc -- which also don't do multi-threaded (or don't do it well and not by default), and which on top of that don't have asynchronous capabilities (again, not by default) and are even less performant than a single Node app.

Which is why a simple Node running with its single process and single threaded execution can e.g. beat a Python server with two dozens of workers (e.g. gunicorn) in handling simultaneous connections (assuming Node code is properly async in the most part).

>And, if the solution is to spin up more servers with access to the same database, doesn't that mean that we are now having multiple threads accessing the database concurrently?

Databases take care of serialization of multiple queries for you -- and for more complex cases (with or without transactions for fuller control).

> Not any more than this is the case with Python, PHP, Rails, etc -- which also don't do multi-threaded

Why isn't PHP etc called single-threaded then? I thought the difference was in fact that PHP fires up new threads for each connection.

>I thought the difference was in fact that PHP fires up new threads for each connection.

PHP doesn't get to decide what happens for each new connection. That's determined upstream, and there's a lot of different pieces of software people choose to do that. Could be directly a webserver like apache's mod_php, or something like fastcgi, php-fpm, etc.

All of those front-ends chose to implement a model of a php process pool. Multiple processes, each with a single php interpreter running. Incoming connections are sent to a process in the pool. So, if the pool is 5 processes, and you get a 6th concurrent connection, that one waits.

However, you can write a C program with a single process and distinct php interpreter per thread. One example: https://github.com/basvanbeek/embed2-sapi

To date, nobody has taken something like that and done the work to connect it to a front end handler.

>Not any more than this is the case with Python, PHP, Rails, etc -- which also don't do multi-threaded (or don't do it well and not by default), and which on top of that don't have asynchronous capabilities (again, not by default) and are even less performant than a single Node app.

uWSGI makes running a threaded or multi-proc python webapp trivially easy (and as of Python v3.6 async comes as standard)...

>Which is why a simple Node running with its single process and single threaded execution can e.g. beat a Python server with two dozens of workers (e.g. gunicorn) in handling simultaneous connections (assuming Node code is properly async in the most part).

Node can beat a threaded python app for sheer volume of concurrent connections to clients, yes. But for a lot of traditional backend work (e.g. talking to a DB) async is no faster (indeed it's often slower) than a threaded approach.

Node (or async in general) is great for terminating inbound client connections; talking to local, in-memory caches or making backend calls to remote, non-local REST services.

For making local DB connections or doing any CPU work (e.g. parsing XML docs returned from an API) single-threaded async rarely yields better performance over threads/multi-proc. A good illustration of this is pgbouncer (async on the client facing end; threaded - i think? - on the db facing side).

Basically, all node really does is reduce the number of front end app servers you need to serve X incoming client connections. Just because node can handle a high concurrent connection count doesn't mean the rest of your backend services can. Regardless of whether connections originate from a single node instance or a large fleet of php/python/rails servers; you still need reverse proxies like pgbouncer/haproxy/twemproxy/squid to manage and shape those connections before they get to things like your DB or internal micro-service APIs.

Because node is single-threaded you also need to keep a very close eye on any CPU bound activity to avoid blocking all your connections. This is not always obvious and can crop up in unexpected ways (see: https://news.ycombinator.com/item?id=15477419)

NodeJS userland is effectively single-threaded, but NodeJS is not single-threaded. NodeJS outsources some event loop scheduling to libuv, and libuv is multi-threaded (by default, 4 threads, but this can be configured. See:

https://medium.com/the-node-js-collection/what-you-should-kn...

Node doesn't eliminate parallelism, it just pushes it outside of JavaScript. A node-based system is parallel at multiple levels: within the process itself library calls can run in parallel, and like you suggest, you have many parallel processes running instances of the same code.

>Does this mean that a web server written in node is running single-threaded?

Yes, which leads to hilarious things like: https://medium.com/walmartlabs/using-electrode-to-improve-re...

"In our tests, however, we found that React’s renderToString() takes quite a while to execute — and since renderToString() is synchronous, the server is blocked while it runs. Every server side render executes renderToString() to build the HTML that the application server will be sending to the browser."

"The average renderToString()call with this configuration took 153.80 ms."

Worth noting that it is possible to have threads without the complications of concurrent modification. You just need to pass things between threads as immutable values. Tcl does it this way (prior to having threads added, it used only an event loop, like JS).

JS webworkers add a sort of multithreading. They force an internal copy, or force a transfer of ownership of anything you postMessage() over.

Reply on news.ycombinator.com