Async/await support in Firefox
blog.nightly.mozilla.org
blog.nightly.mozilla.org
Seven Concurrency Models in Seven Weeks
https://pragprog.com/book/pb7con/seven-concurrency-models-in...
https://www.amazon.com/Seven-Concurrency-Models-Weeks-Progra...
https://www.amazon.com/Programming-Language-Pragmatics-Fourt...
It's pretty easy to read for most programmers who are familiar with a few family of programming languages.
The problem of threads vs async in C seems pretty well studied, but a more interesting question is what we're talking about here: concurrency models in higher level languages: async/await in JavaScript vs. threads in JavaScript. Or let's say Python, because it actually has threads.
I feel like that tradeoff has been less well studied. Interpreters probably use a lot more stack space than native programs, but I wonder if anyone has quantified it.
And on the other hand, the downside of async is less pronounced than in C -- the whole point is to avoid "stack ripping" and explicit state machines.
And I have to echo the recent post here about the complexity of the async/await mechanisms in Python, although honestly I'm not that well-versed in the model.
https://news.ycombinator.com/item?id=12829759
(Interesting that the top comment there is kind of echoing our issue with M:N threading -- the inner platform effect.)
musl is interesting though. I've never seen it pitched as a solution for normal machines. I've always seen it in the context of small embedded systems.
Now when targeting bare metal deployments like unikernels, it is a different story in how to approach it.
Green threads shine because they take ~1kB without tweaking. That means you just pack your software, send elsewhere, and you get those millions of threads per server, instead of losing hours on customer support, and have it revert to 10k threads at random because of bad sysadmins.
Anyway, "thread" is not really a concurrency oriented concept. It mixes so much of parallelism that it's expected that it has some downsides compared o purely concurrent concepts.
One of my pet ideas is just to implement a Go-like CSP with plain pthreads. Channels could be pipes of pointers. Channel select is just select(). No weird M:N runtime needed. I don't want an operating system in my programming language.
This is just stolen from Programming in Lua chapter 30 -- threads and states ( https://www.lua.org/pil/ , not in the 1st online edition unfortunately)
Lua has coroutines within the interpreter, but to utilize all the cores he suggests layering threads and multiple Lua interpreters on top.
IMHO unbuffered channels are one of the most powerful constructs in Go, since they guarantee that "resources" are always on one-side of the channel and are taken care of, and never stored in a channel (or promise, ...) where they could get abondoned/lost. It also allows to make some other assumptions like "the in-memory server has taken my request through a channel and is now working on it and will answer through a chnanel soon" or "the server has shut down so I can't write to the channel", but never "the write to the channel succeeded but nobody cares about it".
Easy. Just implement them with a mutex and a condvar. If you want, layer some lock free algorithm on top for the fast path.
The unbuffered channel would be a good reason to use that scheme. I was thinking of using pipes so Go's select reduces to the select() system call. But I think you can just do both -- it's cheap. Write to a pipe and and notify a condition variable. I'll experiment with it.
Please do it! I feel like we've forgotten the lessons of NPTL, when everyone tried M:N and collectively came to a consensus that M:N wasn't worth it in practice.
It's very compatible with bash so I think it has a chance of being adopted. And I would like to add structured data pipelines, which powershell has. I was thinking of implementing it with the threads and pipes of pointers scheme (and probably the condition variable).
I guess the shell concurrency model is more like a subset of CSP, but the more general CSP model seems useful and fairly easily implementable. I feel like it should be like 200 lines of code, so I should try it sooner rather than later. Just start porting some simple Go programs to it.
( My last post about parsing expressions got buried on HN but I think it is fairly interesting to a specialized audience: http://www.oilshell.org/blog/2016/11/01.html )
If you open source the project, I'd love to try giving you a hand. Anyway, good luck with the project!
I'm rounding the corner on parsing hundreds of thousands of lines of bash scripts now... the prototype is in Python as mentioned in the first post, and the executor isn't complete, but if you want the parse-before-execution, that is working well.
ShellCheck does exist though. IIRC I had mixed experience with it -- it did actually find one bug, but on the other hand it spewed hundreds of warnings about double quoting vars, which is technically true, but not the best use of time for most scripts I write. I'd rather just get rid of stupid quoting rules, which is one of the #1 priorities.
(As far as writing, I find that "omit needless words" from Strunk & White goes a long way. Words like "very" and "a little" somehow spray themselves all over my writing; they are rarely useful and I kill them on editing passes :) )
The more important difference is what kind of synchronization primitives are provided to you by the platform. Go's (synchronous) channels in combination with select provide a quite powerful tool compared to using only pthreads. Even the old WinApi with WaitForMultipleObjects and that stuff will result in other ways or solving typical concurrency problems.
They are really not comparable to threads.
For example, transferables let you move bulk data between threads without cloning. At least that removes most of the communication overhead. https://developer.mozilla.org/en-US/docs/Web/API/Transferabl...
On firefox you can transfer your WebGL canvas to a Web Worker, how is that for useful? https://hacks.mozilla.org/2016/01/webgl-off-the-main-thread/
You can use Web Sockets in Web Workers in very recent Firefox (48+?) and Chrome versions. https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers... And you've always been able to do AJAXy things on Web Workers as well.
I'm not saying the API's are clean or lovely, but the multithreaded functionality is there for the taking.
38+. It's been shipping for a year and a half.
Good to know it's stable for awhile, thanks for the update. Go Mozilla, go!!
I'm not sure what you mean that web sockets aren't supported since the MDN shows it as being fully available:
https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers...
Where this falls down is in handling events where some actions inherently need to be synchronous (preventDefault, stopPropagation), however I think Angular 2 was starting to consider options on how to overcome that. I can't speak for progress in other frameworks.
You can also do XMLHttpRequest and fetch in web workers. And IndexedDB, so you can store stuff persistently from a worker and then read it back from a worker later.
I agree that if you want to hand out data on the client to a worker pool after getting all the data from somewhere in the main page script, then serialization/deserialization can start to bite.
http://research.microsoft.com/en-us/um/people/lamport/pubs/t...
FYI Chrome will ship with async/await support[0] in the next stable version (55, so it's already on Chrome Canary).
https://blogs.igalia.com/compilers/2016/05/23/awaiting-the-f...
This joins generators[1][2][3] (Andy Wingo) and arrow functions[4][5] (Adrian Perez de Castro, Andy Wingo) as examples of the community (read: not employees of Mozilla, Google, Apple, Microsoft) directly working on the engines to ship these features much sooner than they otherwise would be. (Surely they'd land eventually, but acceleration and cross-engine coordination greatly improves time-to-dev-market.)
[1]: https://wingolog.org/archives/2013/05/08/generators-in-v8
[2]: https://wingolog.org/archives/2013/10/07/es6-generators-and-...
[3]: https://wingolog.org/archives/2014/11/14/generators-in-firef...
[4]: https://groups.google.com/forum/#!topic/v8-users/5FNvOv-kQY4
[5]: https://wingolog.org/archives/2015/06/18/arrow-functions-com...
I guess we should just hope that engines will optimize their own promise implementations. This might not matter much in front-end apps, but absolutely does for Node apps.
Other than that, I believe it is time JS gets: 1. An actually useful try/catch (with pattern matching on the catched errors) 2. Standard stack traces across all engines
Those two will really help async/await.
Firefox/Spidermonkey have just ported their Promises completely to C++ [1] which will help performance and enable future optimisations.
But in general, optimizing promises as specified in ES6 is _really_ hard. There's all sorts of work that the spec says should be done which is typically unnecessary (for example, did you know that every time you resolve a Promise with another Promise, that causes creation of a _third_ Promise that no one cares about) but can be observed from script in various ways (e.g. by messing with @@species) so are very hard to optimize out. Promises are also very gc-intensive as specified....
This is one of those cases (iterators being the other poster child) where the usual "yeah, people will just write complicated enough engines with complicated enough JITs to make this all perform OK enough" attitude of the ECMA committee is a bit annoying.
According to this, native Promises are 2x faster than Bluebird in Chrome and 7-8x faster in Safari, and the same speed as Bluebird in Firefox and Edge. I remember a yearish ago they were slow, but that appears to no longer be true.
[1] https://github.com/tc39/proposal-cancelable-promises/blob/ma...
Is there a good way to get Babel/Webpack to emit multiple versions of your code compiled with different features enabled, then load the right bundle?
[1]: https://github.com/Financial-Times/polyfill-service/blob/mas...
i.e.:
async function () { ... }
// or
async () => { ... }Note that that explanation was purposefully ignoring any implementation details about whether or not closures are actually being created/allocated and was purely focusing on whether the programmer had to type out/read closures as extra syntax.
Actually, functional programmers might point out that even without async/await, assignment in imperative languages is still just some syntactic sugar for actually using closures. That is, in functional languages, its almost never idiomatic to directly "assign" to a variable (that is, overwriting the variable's previous value); instead values are "bound" to variables in let statements (or whatever the language's idiomatic equivalent is)--and let statements are effectively syntactic sugar for function calls/closures.
If you're familiar with haskell: you can think of imperative code as haskell's do notation but with the Identity macro (aka no special mode of computation, just the results of each function flowing into the next with "assignment" converted into functions). Marking a function as async (and thus allowing you use to await inside it) is just switching that function to be under the Promise monad. (but be careful! JavaScript promises aren't strictly monadic in that they auto-resolve if nested... so you can't have "Promise Promise a")
(... Sorry, this ended up being a bit of a mess of a braindump because you specified "implicit" closures :) )
1. It allows you to reason linearly about code with IO mixed in, without blocking the entire event loop. This is incredibly useful. Computer programmers are very good at reasoning linearly. It is just much, much more comfortable [1] to think about code that uses async/await than the same code with promises.
2. I think the way that promises and async/await interop is very beautifully designed. The way it ends up working in practice is that you can write a function that "blocks"[2], but if you later decide you want to call it in parallel with something else, that's fine because async functions are just a thin wrapper around Promises. So you can just take that existing function and call `Promise.all()` on it. Similarly if you start with a function that returns a promise, and you decide you want to call it in a blocking way -- you just do `await thingThatReturnsPromise()`. This is an almost perfect example of primitives composing to be more than the sum of their parts.
3. (and this is worth the price of admission alone) error handling works properly and automatically. If you `await` an async function and it throws, you'll get a plain ol' JavaScript error thrown which you catch in the normal way. If I never debug another hung JavaScript app that dropped a rejected promise on the floor it will be too soon.
[1] please don't tell me that I just don't "get" asynchronous programming. I was writing js event handlers when you were in diapers (something something lawn something).
[2] `await` doesn't really block the event loop, it just appears to.
This sounds pretentious and detracts from an otherwise good comment, even if you didn’t mean it to.
Makes writing correct code easier.
With async/await the language takes care of lowering those control structures that straddle async operations, similarly a compiler lowering them into conditional branch instructions.
However if you have control flow in it (execute this async function 100 times, add the results, depending on the result call another async function, ...) things will get a lot more ugly with raw promises.
Allowing programmers to write code as if it was synchronous removes a lot of confusion, boilerplate, and bugs.
Nothing was impossible. Maybe if you had 3+ levels of nested Ajax calls then it needed bit of head scratching, but jQuery's Deferred and other promise libraries are pretty neat to solve these issues.
However this could be indeed hidden from the programmer.
Look, I get its use case. It simplifies doing asynchronous and synchronous programming together. It takes an asynchronous operation and turns it into synchronous which worries me a bit about new developers and code readability. There are APIs including parts of node, HTML 5, etc that async and await isn't going to work with. So now you're going to have normal synchronous code, asynchronous code that looks synchronous, and other asynchronous code that can't be made to look synchronous.
I'm all for finding ways to avoid callback soup and to simplify asynchronous patterns. I think there are good ways to structure traditional callbacks that is sane and I think promises are a great step. I'm skeptical about async and await.
XMLHttpRequest was enhanced with the Fetch API which uses promises. The libraries and methods will eventually catch up in a similar way.
If we stopped here there'd be little reason to ask everyone to promisify their libraries. The promise callback chain was only slightly more easy to read than the callback pyramid of doom. You might say that async/await lowers the explicit complexity vastly and increases the implicit complexity marginally.
Too bad promises had a lot of glaring flaws that are taking forever to fix (or are unfixable). We're just getting to the point where unhandled errors are dealt with properly. Composing promises with non-promise constructs still suck. There's still only 2 paths (success and error). The flatMapLatest scenario still suck.
Still, promises were the improvement. Async/await is just sugar.
This is just pointless syntax.
That said, async functions also introduce a fundamental language shift in how one parses JS. One can now block execution to return a value, but no longer know if the innards of a function is blocking, which could mean that your function execution is blocked by an async execution. Instead of what one may know as async by looking at a promise or any other similar construct for handling async flow like observables, this will be the source of many bugs in developers' code as we shift in mindset, should this become popular. This would definitely improve readability of code written using continuation passing style though.
Minus the awful try-catch, I'm not necessarily against async-await being the norm, but its broad effects on how async code will be written should be recognized, especially considering that it doesn't solve a fundamental issue like something like observables do.
const maybe = await fetch(url).catch(err => err);
const hits = await cache.incr(key).catch(err => 0);More likely you're arguing people shouldn't use 'exceptional situation' error handling as often as they do... but that's a separate argument. I think going from 2-3 syntaxes for the same thing to 1 syntax is strictly an improvement, especially since huge amounts of promise code silently swallow errors when people don't properly chain promises or handle the .catch case.
To catch thrown errors, not to implement the behaviour of transforming rejected promises.
Huh? I think you're misunderstanding how these functions work. You can't call an async function from a normal function and have it block the normal function. You specifically have to await the Promise returned by any async function.
I just wanted to add (may help others): And you can only do that from within an async context (currently just the body of an async function).
> I have one major misgiving about async-await - the regression in needing to do a try-catch if one isn't isn't a test situation. This is a terrible syntax to have to write to handle this.
If you don't like try/catch, you can still do:
await getPromise().catch(err => {/* handle error */ })
const username = do {
try { await getUsername() }
catch (err) { 'unknown-username' }
}
> One can now block execution to return a value, but no longer know if the innards of a function is blocking, which could mean that your function execution is blocked by an async execution.
I'm not sure what you mean by that but you still have to explicitly use the "async" keyword for any function that uses "await". And function calls that are not prefixed with "await" will not block just like before. In fact, in an async/await codebase, you can more easily tell which functions are async because they'll have "async" in front of them (unlike errback/promise code which would require you to read the body of the function).
> Minus the awful try-catch, I'm not necessarily against async-await being the norm, but its broad effects on how async code will be written should be recognized, especially considering that it doesn't solve a fundamental issue like something like observables do.
Yes, it's basically just syntax sugar for promises but it does remove a lot of boilerplate code and is easier for JavaScript engines to optimise.
[0] http://wiki.ecmascript.org/doku.php?id=strawman:do_expressio...
I feel like it's less difficult to shoot yourself in the foot with Python's implementation, because you know that if you're calling an asynchronous function, you have to `await` it or it won't work and won't have side effects. In JS, you could call the async function without awaiting it and it could have side effects which could produce subtle bugs that aren't easy to find.
I personally prefer the functional approach with bound methods to avoid callback hell, improve readability and ensure there is minimal amount of memory leaking.
That being said, async/await makes callback programming easier so it is another tool which may come handy in the situations where you really need it.
Think of async as return and await as bind.
I hear the words you are saying, I just cant visualise the concept. Could you ELI5, or post an example?
I've been working to teach Promise thinking to another developer. Promises (and callbacks) are "easy" from a functional programming background, but for someone without much of a functional programming background like my colleague, it's certainly confusing. There is a rigor needed in writing Promises (and callbacks) in knowing what is in each closure and making sure that the return values of individual callbacks are in the "right shape" for the next callback in the chain.
Loops with Promises involve confusing things like higher level combinators like Promise.all and Promise.race, and that requires more new sorts of reasoning about return types than a "traditional" `for (let thing of list) { let result = await doThe(thing); /* do something with result */ }` loop.
Not that a need for things like Promise.all and Promise.race goes away with async/await but that it moves from being a must need to learn on the first pass of writing an algorithm out to being a performance optimization in advanced scenarios that can be done easier by someone more senior and/or in a later pass of code writing (such as a code or performance review).
It makes the learning curve to "doing asynchronous code right" a lot less sharp, smoothing out some of the complexity, and that can be a huge win for any team with a mixture of developers of different skill levels and skill sets.
If you wind up writing the word "await" in your codebase more than once, you're doing it wrong. You will quickly see how futures/promises wind up taking over all of your code. This is a very good thing. You shouldn't need to block, ever.
Sincerely,
Scala coders
Why have you decided to rewrite everything?
Sincerely, Everyone else. (Obviously not JavaScript coders, sadly...)
P.s. I can only hope you have a somewhat sane release story now.
It makes code more readable and easier to write, but doesn't fundamentally change how it is working under the hood, as far as I am aware.
var txt = await read();
// This code is blocked until read() completes.
console.log(txt); read((err, txt) => {
console.log(txt)
})
read().then((txt) => console.log(txt))
async function is blocked on await statement, after all that's why it has such a name.
It does not block the whole VM but task (async function) is effectively blocked for an observer inside it.
http://docs.scala-lang.org/overviews/core/futures.html#block...
Also, how is Scala doing it?