Is it just me, or is this not at all the same kind of node.js-style concurrency that makes sense Heroku's new router? Using Unicorn simply means that you're firing up a few more workers, it's only slightly better than buying more dynos.
It is probably about the same as buying more dynos -- I would think that unicorn set to fork three times is actually about the same as three dynos with thin in standard configuration -- as you say, either way it's three processes, each of which can only handle one request at a time.
It is _probably_ not a solution to the bad worst-end latencies that can occur when you have random (or round-robin) routing, and relatively high variation in request durations.
Using a multi-threaded request dispatch (NOT what unicorn does, but what they start hinting at towards the end) _may_ be a solution to that though. Although multi-threaded concurrency is STILL not "at all the same kind of node.js-style concurrency", as node.js is evented, not multi-threaded. But multi-threaded concurrency seems likely to make sense with random or round robin routing too. Although more research and data is called for (at least more than I have, which is zero).
But there's no reason to assume that node.js-style evented concurrency is the only thing that can possibly make sense with random routing. And if it is, that's very inconvenient. Trying to do node.js-style evented concurrency with a Rails app... or any app that wasn't built from the ground-up (including all it's dependencies) for evented-style concurrency... or even if it WAS... it's non-trivial.
In contrast, multi-threaded request dispatch requires not a lot of change to your code -- basically just avoiding shared in-memory state between requests (basically, global/class variable access).
In addition to puma, thin in fact does have a multi-threaded mode, although it's poorly documented.
I think they are actually passing the buck suggesting "We can't tell you which way to do multi-threaded request handling, it depends on your app!" Well, sure, it always depends on your app, but that applies to their suggestions to use unicorn as well, but that didn't stop them. There are still often general best practices for typical web apps.
I think it's true that they can't tell us the right way to do multi-threaded request handling -- because it requires more research and analysis and experimentation (and possibly a patch or two) to figure out the best practices here, the community hasn't put enough into it yet. But if anyone's got the resources to put into it, it's heroku, and it would be rather good for their business to figure this out and educate the community. Heroku became so respected because they seemed to really know what they were doing, to be at the top of the game -- if they can't manage to figure this out either, it lowers our trust in them.