Taming the Asynchronous Beast with CSP in JavaScript | Hacker News Reader

Taming the Asynchronous Beast with CSP in JavaScript

jlongster.com

147 pointsjlongster29 comments

He mentions using CSP with JavaScript:

> It's safe to say that it's becoming quite popular (and validated) and I think we need to try it out in JavaScript.

To be clear, and he mentions that Clojure has core.async, it's been done in JavaScript (if you will) already via ClojureScript.

David Nolen has done a series of posts on it starting at [1].

[1] -- http://swannodette.github.io/2013/07/12/communicating-sequen...

EDIT: And I should have finished reading the post before posting this. He does mention David Nolen's series of posts on ClojureScript and core.async

What would be nice as an addendum to this, especially with the likes of browserify and webpack for node and the browser would be a syntax to start an async channel in a new process/webworker...

    var ch = new echan('moduleName');

Where the moduleName would be a module that exports a single function that receives two channels.. in, and out for a request/response pattern. An "easy" level of abstraction here where the request/response is only going to pass what works with JSON.stringify/parse, removing circular references.

It could work really well for worker handlers... It only needs to be an abstraction around the csp interfaces, as well as the current implementations for web workers and forked processes in node.js

More advanced scenarios could use 0mq for passing a worker to multiple processes... all of that said, it's just an idea.

I think where this lines up very cleanly is as a means of communication with React and flux.

I translated the pingpong example to use events instead of CSP. https://gist.github.com/aynik/94c9f3d01139986d2b91

Even mentioned transducers, cool! But, I was hoping for a treatment of CSP and react/flux. jlongster teased us about that here[1]. Maybe the next post in the series? (plz?)

1. https://justin.harmonize.fm/development/2014/08/05/om-and-fl...

My understanding of Javascript is that it is essentially single-threaded. Maybe I am mistaken, but if that is indeed the case, how does something like JS-CSP or Clojurescript's core.async accomplish concurrency? Has there been a change in the underlying Javascript? Is it just using tricks to make single-thread act like multi-thread?

Short answer: Yes.

Concurrency does not necessarily mean parallelism (multi-threading). CSP-derived mechanisms provide abstractions for units-of-work to yield control to other units-of-work. Here, a unit of work is a function invocation, but it can be a block, a thread-like object etc. With such primitives it is fairly easy to build cooperative (as opposed to pre-emptive) concurrency mechanisms.

For a mature example, see e.g. http://www.stackless.com/

In the JavaScript "user space" you don't have access to threads (with the exception of web workers), but the JS engines themselves are multi-threaded and fully concurrent. This is why your UI doesn't freeze when doing XHR (assuming you're not doing synchronous XHR) and represents the need for elegantly handling asynchronicity.

Say you have:

  function one() {
    console.log("one");
    setTimeout(one, 1000);
  }
  
  function two() {
    console.log("two");
    setTimeout(two, 2000);
  }
  
  one();
  two();

They are executing concurrently, but never in parallel (since javascript is single threaded).

Now, how do you coordinate them if you need to? This is what CSP helps you do.

I've spent a fair amount of time in core.async, and I feel like I'm on the edge of enlightenment here, but I don't quite understand how CSP helps with this example. Is the point that you can take multiple puts off the channel in one go? That's the only thing I can think of.

Maybe you could present the CSP alternative, and a basic example use case?

* edit: I guess a basic example is a sliding-buffer that only keeps, and thus only reacts to, the last mouse event. Is this on the right track?

The example was more about showing that concurrency and threads are not synonymous, but you can imagine one of the functions producing a stream of values and the other waiting for and consuming them.

In vanilla javascript, you have to resort to breaking your code up into callbacks, which looks trivial in a two function example but becomes unwieldy once you add a few layers and do error handling.

With CSP you organize your code around channels according to the code's logic, rather than according to the needs of coordination. This makes things like error handling and reasoning about your logic much simpler.

I totally agree that it helps with callback hell / reasoning, but I think your argument was more about coordination. Hence the sliding buffer example - something that would be very, very hard to /coordinate/ with callbacks.

Why doesn't he state what CSP is upfront, requiring the reader to visit Wikipedia to know what it is?

Buzz words and acronyms are getting out of hand on the web.

How does this compare to FRP style ?

It's more general than FRP so is better at stuff like simple async coordination (what most people use promises for). But you can easy do FRP-style stuff on top of channels, which is basically what transducers are. So they are closely related, but have different roots.

Rather than saying that CSP is more general, I'd say CSP is more provides you a more low level constructs. In FRP signals are the key concept that represent value that changes over time. Signals have a pushed based interface that's kind of why it's reactive. FRP typically provides pure functional interface, meaning you never put values or emit events, you just have control flow structures that allow you define transformations over base signals. FRP signals can be easily implemented via CSP channels, but not other way round, that's because CSP channels are push based for the producer side and pull based from the consumer side.

If I had to make an analogy channels would be queues while signals would be an observable variable.

You've got to wonder why exactly we still (in 2014) have to contort ourselves around a single threaded model. So that we end up with this level of complexity.

JavaScript is great with data but is becoming a monstrosity when it comes to concurrency.

Can we all stop and have a deep look at how other languages deal with concurrency. I have (like many of you) used many different languages and have played with several different concurrency models. The bottom line for me is that multiple threads + concurrency frameworks (i.e. no exposed concurrency primitives) are still the best solution.

I appreciate JavaScript is trying to avoid deadlocking etc. but in modern concurrency frameworks this just isn't an issue.

Seriously we're now contorting ourselves around dogma. Please browser folk, come up with a decent solution to save us all of this.

VOTED DOWN FOR HERESY :-)

I think you've completely missed the point. CSP channels are in fact what are used for elegant parallelism on Clojure. The processes are backed by a thread-pool and can be run concurrently. Whether or not it's backed by real threads or not is completely opaque.

Do you feel that:

var ch = chan();

go(function() { var val; while((val = yield take(ch)) !== csp.CLOSED) { console.log(val); } });

go(function() { yield put(ch, 1); yield take(timeout(1000)); yield put(ch, 2); ch.close(); });

Is clean and easy to understand. That's the point I'm making is that this contorting around a problem. I don't want my code littered with all this annotation just so that it can be synchronous.

Also I'm not picking on the one solution, I just want everyone to take a step back and see if we're really heading in the right direction here.

It's worth stepping back, and it's far easier than the alternatives for sure. (note that with macros you could even have nicer syntax).

But yeah, if you're arguing for real threads, that's a different discussion. Even so, you'll have to do similar coordination between threads which will look similar.

Unfortunately, trying to add true threading to the browser in Javascript is so, so far in the direction of "insane" that it's not an option, so we're sort of stuck in the browser.

Note that I am deliberately referencing that whole set of qualifications: "true threading in the browser in Javascript". Drop any chunk of that and it becomes less insane; for instance, WebWorkers drops "true" and gives a very restricted version of threading. Threading JS in general wouldn't be any harder or easier than threading any other dynamic language in theory (though in general those sorts of languages have a really poor track record, but only in practice, not in theory [1]), but adding "in the browser" is a big problem. Maybe if Servo works properly it will be feasible, but until then it would just be a nightmare.

But I would certainly agree that if you have a choice, such as on the backend server, and you choose Javascript for concurrency, you're making a very poor choice. It just isn't what the language was meant for.

[1]: As near as I can tell, they all tried to retrofit threading onto themselves a very long ways into their development (many years if not decades after birth), and the result- vary from "failure" to "barely functional". However if one set out to create a new language in the dynamic family and made threading a concern from day 1 there's no reason I can see you couldn't succeed. Elixir would be a sort of existence proof. It's the retrofitting that failed moreso than the general idea.

not quite as much... though using "async" as the main method name, with the fat-arrow syntax for an anonymous function would be a little more clear... same for using let instead of var here.

    async(()=>{
        ...
    });

It's a bit more terse, and implies that the method will be run asynchronously from the parent.

It's not clear to me that you actually read the article since a large part of it is about elegantly dealing with streams of UI events such as mouse movement without resorting to callbacks. Not sure how multithreading even applies here. But if you want to talk about multithreading the model described is exactly the one embraced by a new up and coming language called Go - it's users love this stuff precisely because it's more readable than writing code with explicit threads and concurrent queues.

So I'll avoid the potential language flame war as best as I can.

It is the sync on async part I am talking about (see code snippet in sub thread). The layering of synchronous constructs on an asynchronous model which only exists to support the single threaded browser model is getting very convoluted. That's the part I'm talking about.

It is perfectly reasonable and trivial to support a multithreaded model in the browser if the multi-threaded code does not alter shared state. Which obviously is the intention of Web Workers and in there case there state seems to be primarily the DOM, I just feel that this is the direction we and browser developers should be looking into more and where the effort needs to be. And that Web Workers are part of the story not the story.

In my experience concurrency can be kept to a remarkably simple level if either the developer or the language/framework author are willing to follow some simple constraints. In my ideal world:

     var x= concurrent(function(x) {
         return doSomething(x);
     })(1000);

Would execute the function without any closure or access to globals (like the DOM). Now we have no shared state, so we could run this function in a parallel thread. Now add back access to stateless resources like HTTP requests and we're rocking and rolling.

Web Workers just seem to be a heavy handed approach to achieving this.

If there is a framework or method available to achieve this and I have missed it out of ignorance please someone correct me and show how this can be done simply. My goal is to have simpler code not to win an argument.

What solution would you recommend? I have given this a lot of thought and have a few ideas. Is there a venue for discussing the problem and suggesting possible solutions?

In lieu of a decent forum. I'll drop my thoughts here.

I have felt that the fundamental problem with JavaScript is that it cannot pre-empt. Even with its concurrency of web workers the primary example they give shows the flaw inherent in their solution

http://www.whatwg.org/specs/web-apps/current-work/multipage/...

As a challenge, try and write a version of this worker that the owner page can switch between two number sequence algorithms(prime / Fibonacci for example). The only way you can do this is to rework the program until it's back to being a program beholden to events.

You could solve most of this with a pre-emption mechanism. Allow an additional way to respond to certain inputs by registering an interrupt instead of listening to an event.

    function myInterrupt(preEmptedState) {
        interruptCounter+=1;
        //do something useful here
        setExecutionState(preEmptedState);  //switch execution context back to pre-empted thread.
    }

The parameter to the interrupt function is the execution state of the thread that was pre-empted. This can be restarted at the end of the interrupt, stored for continuation at a later time (enabling blocking), or abandoned (effectively terminating the thread).

What this would need is:

An execution state object, possibly an opaque object.

addXXXInterruptHandler(interruptHandler) ;; XXX= Timer/Message/Input/etc.

setExecutionState(newState) ;; this function would not return, it would continue the new state.

createExecutionState(main) ;; generate a new execution state object ready to run the function main

triggerInterrupt(interruptType) ;; Allow a tread to trigger an interrupt upon itself.

If interrupts themselves are simply the running of a new execution state then TriggerInterrupt is just a special case of setInterruptState.

Having this as a core would allow a huge number of higher level constructs that would enable blocking functions, and sane interactions with running processes like web workers.

Yes, you would introduce the possibility of race conditions etc. but only if you actually used the new functionality. If you chose to leave it alone you would be on the old single eventy thready system that we know and hate.

It would also mean you could have someone write var x=0; while(x<10) {} without it killing all scripts.

Interesting, I'd suggest that your solution would need to simplified dramatically to make it accessible or to be more under the hood. There is still a bit of room to shoot yourself in the foot.

In terms of my suggestion it is simply that we need a language construct that creates a new scope without access to globals or 'this' and creates a pass by reference (i.e. deep cloned) closure. This allows for zero shared state. Then add back in access to no-shared state globals like HTTP access.

Think web workers on a micro not macro scale.

If you add shared memory threads (like in c++, c#, java) to Javascript, I'm disabling it forever.

funny, i read that as CPS and it still made sense.

Same here.

Me too.

Reply on news.ycombinator.com