Why HTTP Streaming? | Hacker News Reader

Why HTTP Streaming?

weblog.rubyonrails.org

84 pointsab933 comments

I've yet to see my concerns properly discussed (how to handle exceptions): http://news.ycombinator.com/item?id=1671437

I believe that's one of the main reasons this won't be on by default.

It's going to be a tradeoff, either you accept that an exception can occur after sending a '200 OK' status code and possibly resort to javascript to display the error in a nice way, or you'll use the old fashioned way and send the correct response when the full page has loaded.

I'm going to use HTTP Streaming and rely on my tests and Hoptoad to fix errors as soon as they'll occur. I think the performance boost outweighs the consequence of not sending the correct status code when an exception occurs.

We won't do anything to handle exceptions in 3.1. For 3.1, this is definitely a "use at your own risk" feature. Besides handling exceptions, there are a bunch of caveats for using this. You need to use the right web server (unicorn or specially configured passenger), along with the right proxy (nginx or apache with special configuration) and Ruby 1.9.

I've got ideas for handling exceptions, but I just can't make it happen for 3.1. :-(

This is the first thing that came to mind for me. I recently wrote a WSGI Python app that made use of chunked encoding, so that it could emit multi-hundred MB text files without loading them all into memory first, and one of the big hurdles was running into an error after having sent some data. In the end I had to implement a fancy Javascript check and keep some state in the app to check if there had been an error and the downloaded file was no good.

It's funny; some of the last work I did in Ruby was getting streaming support into Rack nearly three years ago. http://technomancy.us/109 It's a shame to see it took so long to make it to Rails.

Besides asset fetching, do any browsers start rendering pages as chunks come in?

EDIT TO ADD: ok, so how does this work if you're page has js/jquery code executing on load? (say for js-based navigation and layouts) ... are people gonna see funky stuff before the page is fully loaded and the js is executed?

Yes I believe so. If you want to try this out you can setup a PHP page that puts some content up and then calls sleep(). You'll see that the content that has been processed and sent to the browser is displayed as the rest is processed.

To do this you probably want to ensure that the webserver output buffer, PHP's output buffer, and possibly even the TCP stack buffer (if they are all different sizes) are filled with enough test data to get flushed and sent to the browser before calling sleep(), otherwise you won't see much in the client.

On a slightly related note tuning these 3 values and setting them to sizes appropriate for the content you are serving can increase your webservers performance.

flush()

Absolutely.

They won't do it for application/xhtml+xml, so if you want to see this in action, try sending your page as text/html and then application/xhtml+xml. You'll notice that the XML-based page seems glacial.

(Why does this happen? Who knows. I have been using streaming XML parsers since like 1990...)

I believe its because thanks to the structure of XHTML, you cannot validate a page until all the content has been received.

There are streaming XML validators.

But who cares? The browser isn't validating HTML as it's rendering it, otherwise the internet would be one big blank page.

I think that's the point: Browsers don't bother to validate HTML, but they have to validate XHTML to know whether to pitch a fit about errors as required by the spec.

Browsers? Spec? What universe are you from! :)

Please note that there is an important difference between "well-formed" and "valid".

> But who cares? The browser isn't validating HTML as it's rendering it, otherwise the internet would be one big blank page.

But that's the big difference isn't it? Browsers fix html as they go, but they blow up on XML (as mandated by the XML spec).

What would be weirder, seeing the page either render fully or blow up, or seeing a page start rendering and then at some point be replaced by an error message?

Of course ideally it would start rendering and keep rendering, never blowing up, but that's not really allowed in XML.

> What would be weirder, seeing the page either render fully or blow up, or seeing a page start rendering and then at some point be replaced by an error message?

Who says the page has to be replaced? Webkit and Opera (IIRC) happily render the page up to the first error that they encounter.

Gecko's decision to do the YSOD was a bad one.

> Who says the page has to be replaced? Webkit and Opera (IIRC) happily render the page up to the first error that they encounter.

That shocks me quite a bit. This definitely didn't use to be the case for Safari, early versions implemented draconian XML error handling and refused to display non-well-formed documents.

> Gecko's decision to do the YSOD was a bad one.

Meh.

It's still draconian error handling, all the spec says is that you have to stop parsing and notify the user.

> Meh.

There were many circumstances that prevented the adoption of XHTML-as-XHTML. One of them was the fact that your users would get a cryptic error page, with no context, if something went wrong.

What Webkit does is much less threatening.

Yes, I've actually used it before to set up a hacky progress report for batch tasks.

I just set up a node.js app recently because it does chunking by default. I noticed Chrome didn't want to render the early chunks. I believe some folks mention that some browsers will only render the chunks if a large enough buffer of chunks come down. But Firefox gladly rendered the early chunks. It's easy enough to setup a response.write("Hello") setTimeout response.end("World") in node to see the chunk rendering in different browsers.

This is sometimes handled at a level lower than than the browser (in the socket APIs browsers are built on). Two or three years ago, anyway, when I was last working on this stuff, the buffer amount in Safari for example depended on OS and OS version. The solution is to just throw half a kilobyte of nothing down the wire before starting to send chunks.

Has anyone tested how much of a performance benefit is this likely to give? It seems that it would benefit you most if you have a slow page composition layer but generate a lot of static includes like js and css. Arguably in that case you should be using ajax to retrieve the slower bits, or server side caching instead. But maybe i am missing a use case or some measurements that show it is more generally applicable.

Can anyone comment on omitting Content-Length in the response, versus using chunked encoding? Will chunked encoding simply give you more control over when the browser executes the content, or perhaps be more compatible with any proxies in between? Or is it just a matter of being able to reuse the connection afterwards, rather than having to close it?

Using a simple CGI script, both methods achieve the same and work in FF and Chrome - chunks sent have their script statements executed, so you can e.g update a progress bar as you render partial content. However, I had trouble getting it to work with gzip; I had to turn off gzip (SetEnv no-gzip in .htaccess) otherwise the whole output was sent at once (this has possible to do with some default compression buffer size setting).

How did it work before with RoR? The whole response was buffered and then sent at once with a Content-Length header?

I wonder why "HTTP Streaming" (known since years as "Chunked Encoding") is such a big deal now.

Not sure, but I know for myself it's because of node.js and a need for a better realtime app. I couldn't have told you what Chunked Encoding was before a couple weeks ago when I started fooling with node.js. Realtime apps are more popular, so node.js is more popular, so now people know about Chunked Encoding and Keep-Alives?

I don't think they're doing chunked encoding. I think they're making their template engine emit a stream instead of emitting the rendered page. Then, each time the template system has some text to send, it's written to the network rather than accumulated in a buffer that's all written at once.

The innovation is not what headers to send, it's to produce your data incrementally instead of all at once.

What I meant to write was that I don't think the encoding is what matters; incremental writes to tcp sockets are easy, and http is not much harder. What's hard is making your application code work well with this.

Quick question: Is this suitable for streaming video? Just wondering, thanks.

There isn't really much benefit of using chunked encoding for streaming video, if you're not generating the video on the fly server-side.

If you are simply streaming a stored video, you can just get the file size, use that as the content-length, and send the video. If the content is in a streamable format the client can just read/play the data as it is received. Chunked encoding will in fact add a little overhead to just sending as is.

Chunked encoding is more apt for situations where you don't know beforehand how much data you are going to send.

Yes, this is how Move Network's streaming works, how Apple's streaming works, and Adobe just announced official support for streaming this way last week[1] (edit: oh and microsoft buit it into iis too).

What they do is parallel-encode their video feed to many different bitrates, then as the client is keeping up or not keeping up with the incoming chunks they move up or down the scale of which chunks to send. The best part is, it works via existing http cdn's like level3 and akamai, so just about anybody can stream live video this way to as many people on the internet as they can get to watch it for the already commodity cost of cdn bandwidth.

[1] - http://arstechnica.com/apple/news/2011/04/adobe-throws-in-to...

This is a common misconception. In fact, HTTP Live Streaming as proposed by Apple has nothing to do with chunked encoding. The former splits the video into multiple "chunk" files, using an m3u file as a playlist. Stream variants are supported by having multiple sets of video and m3u files, but dynamically switching between them based on bandwidth is left to the client.

See:

http://tools.ietf.org/html/draft-pantos-http-live-streaming-...

Not really. Streaming video tends to use byte-range requests (introduced in HTTP 1.1) because you want be able to to drag the video play head/cursor to an arbitrary location. For example, iOS uses this mechanism.

Reply on news.ycombinator.com