Why your web framework should not adopt Rack API
blog.plataformatec.com.br
blog.plataformatec.com.br
I'm the maintainer of a similar project for clojure called noir-async. Noir is built on ring, clojure's version of rack, and has no notion of streaming responses, but adding it in wasn't a problem at all (https://github.com/andrewvc/noir-async).
In other words, it's not a big deal. Most streaming code should really be a separate app anyway. Apps that deliver true streams probably need different infrastructure from say a rails app. You want a separate process running a reactor to handle concurrent IO.
Of course non-blocking is what people want for stuff like websockets in most situations.
A great deal of the reason you are saying that is because it's too hard to do anything else today. That doesn't make it the right answer.
It is far easier to take a fundamentally-streaming stack and layer conventional request-response on top as a special case than it is to take a conventional request-response stack and layer a stream on top of that. It isn't even that much harder to do it right if you start with that from day 1, but I've had a devil of a time convincing anyone in the early phases of web framework development of that. ("Why don't you do it yourself?" I'm full up for spare time projects at the moment... and also, I'm really trying to resist adding yet another web framework to the world.)
Of course sometimes the right answer really is a separate app. But you should do that because it's the right answer, not because your stack forced you to do it that way.
Exactly my thoughts.
This is demonstrably true in that Rack/etc are themselves almost always built on top of conventional socket-based streaming APIs. Streaming APIs have been the norm, and yet people choose to put abstractions on top of them. Why? Mostly because it gives substantial improvements to application code, and turns the system into more of a set of contracts and less of a mushy touch-any-object-you-want system, which I think is fundamental to how streaming works, due to its imperative nature.
Using Rack or any of its equivalents, it would be fairly easy to allow a hole to be punched through the system to give a direct socket streaming API. Just toss the relevant objects into the request. It would violate the abstractions of Rack, any middleware, etc. But you can do it, and it would be easy. It's not unlike using Rack instead of the abstractions Rails puts on top of it – arguably it can be useful shortcut parts of the stack.
Why don't people do this? Mostly because no one cares to. It's not that useful or interesting or testable. It's noticeable enough that people complain about it, but not noticeable enough to be worth it to take the hit and use a more imperative API.
The idea that streaming can't have abstractions put on it is beyond bizarre.
You're asking the wrong question. The right question is how much of a problem is it for your conventional request-response app to be built on top of something that really wants the streaming aspects exposed. And the answer is quite a bit.
For example, there are significant performance benefits to putting caching proxies in front of applications. But whether that is a reverse proxy sitting at your hosting facility or a distributed proxy that you're getting from Akamai, they introduce a strong assumption that you're not going to stream content through them.
So stuff you're going to likely want to do anyways at some point pushes you to want to separate request-response from streaming. And if are going to make that choice anyways, it makes little sense to add any complexity to your life at all to try and keep those together.
Of course you want a strong page framework built on top, but the point really is it's easier to start with the streams underneath and build something that limits the streams to the request-response cycle than to go the other way, not that request-response isn't an important simplification that is frequently useful or that it shouldn't be very strong itself.
But you shouldn't go and write it into your WSGI standard or hard-wire your shiny new web server to insist on what are morally CGI pages, and break down crying if somebody wants to stream a little bit.
In any case, the main point of the blog post is the amount of effort required in wrapping existing Rack middleware so they play along with streaming but still in a non-optimal way. For example, see the connection management middleware in Active Record:
https://github.com/rails/rails/blob/3-2-stable/activerecord/...
Even more, if your rack-stream middleware is after the connection management middleware, each stream loop will first need to pass through the proxy call. The longer your stack, more proxies it will likely have and more cost will be involved in each stream.
So yes, you can work around but requires a lot of extra work and is not as efficient.
However, the reason I wrote that non-streaming endpoints should be separate apps is that their performance profile tends to be radically different, and you probably want those concerns separated.
For those interested, Aaron Patterson (tenderlove) talks about this in his RailsConf talk, starting at 26:45.