Hacker News Reader

Comment by cromwellian | Hacker News Reader

HTTP itself has an overhead, I'm skeptical changing a persistent IMAP connection into a request/response model with a different serialization format is going to be more efficient, especially given compression (like COMPRESS=DEFLATE)

IMAP4rev1 is one thing, but there are many extensions supported in modern imap servers to speed up syncing and mailing, like CATENATE/BURL/CONDSTORE/MULTIAPPEND/MOVE/etc

Granted, the IMAP protocol is pretty hairy and difficult to work with. On the other hand, there's a huge ginormous deployment of it on the client and server, and the IETF WG behind it. I doubt JMAP will replace it anytime soon unless the WG itself takes up the issue. And given the neckbeards there ;-), some of whom have been working in IMAP for decades, you face the same opposition to change as those trying to legalize drugs or gay marriage. ;-)

We're still trying to get IPv6 deployed!

View on HN

> I doubt JMAP will replace it anytime soon unless the WG itself takes up the issue.

Maybe. I'm curious what the effect would be if a single mobile platform provider (iOS, Android) supported a new protocol with a single main free mail provider. Specifically, Gmail and Android could possibly push major change through.

Look at SPDY, it was originally an experiment with Chrome and only a subset of Google services, and is now being deployed to other properties and iterated on rapidly in other browsers, too.

That's why a major focus of JMAP is to be able to be put in front of an IMAP server with a proxy (and it takes advantage of CONDSTORE if present - we built from an IMAP base)

Honestly, if it's not significantly nicer to work with than IMAP, it won't see traction, and that's fine.

Mind you - MOVE took bloody long enough in IMAP, I was arguing for it for years before it finally got support behind it. There's still not much support for SUBMIT via IMAP though, which means multiple connections via different protocols to do basic emailing, with all the support fun that entails.

Anyway - worst case, we still have a better protocol for our own stuff, and that's not nothing.

HTTP can be made persistent too, both with good ol' KeepAlive and with recent innovations like SPDY. Massive savings in latency, especially when SSL is involved.

One of the main efficiency gains is in massively reducing the number of round trips required to do a series of operations. This batching of operations is hugely important on high-latency connections, relevant for us in Australia all the time but also of importance on mobile connections the world over. Even 4G networks have relatively high latency, but can do a burst transfer pretty efficiently.

A binary protocol running over persistent socket would have fewer round trips and parsing latency. HTTP and SPDY can't compete with that, because they introduce too much header overhead and complexity.

Persistent connections are tough on patchy mobile networks.

Parsing latency isn't really an issue; all this stuff is IO bound anyway. CPU cycles are cheap.

If persistent connections are patchy, then HTTP is no further ahead.

A sure-fire way to decrease latency is to send as little as possible.

An empty default browser HTTP request is likely to cost you around 500 bytes in headers, before you have added any headers or data. Contrast that with a binary command which can be 1 byte. And then multiply this by hundreds of requests. CPU cycles on mobile are not cheap. It makes no sense to parse 500 bytes (and then GC this later) when you don't have to.

Even on servers, parsing isn't cheap. Look at nginx code or other high-performance parsers. All sorts of little tricks to compare 4 or 8 bytes at a time to determine the verb and whatnot.

(Author of the spec here)

The protocol could just as easily go over a WebSocket as HTTP (you'd have to change authentication mechanism, but that would be trivial). The only reason we don't do that at the moment is: 1. Browser support (we support back to IE8, unfortunately) 2. Lack of GZIP compression (particularly of responses from the server); this was being worked on last time I looked, but I haven't checked recently to see if it's been implemented yet. This is crucial for saving bandwidth.

That would get rid of your HTTP overhead; I will put some mention of that in the spec (I think HTTP support should be required, and WebSocket support optional).

As for JSON vs. binary: * There are libraries to use JSON everywhere – binary protocols would require everyone to implement another parser, adding extra difficulty for adoption and a surefire source of bugs. * JSON is readable over the wire, binary is not. Trust me, in the real world, this is a huge advantage for debugging all sorts of things. * JSON can be used trivially in a web app. Apparently this World Wide Web thing might catch on. * The parsing overhead of JSON is not too large; yes it's longer than a binary protocol, but it hasn't been a bottleneck for us, even on mobile.

> One of the main efficiency gains is in massively reducing the number of round trips required to do a series of operations.

Er…

> 5.5. Multiple Commands in Progress

> The client MAY send another command without waiting for the completion result response of a command, subject to ambiguity rules (see below) and flow control constraints on the underlying data stream. Similarly, a server MAY begin processing another command before processing the current command to completion, subject to ambiguity rules.

IMAP already supports batching.

Reply on news.ycombinator.com