Comment by Sir_Cmpwn | Hacker News Reader

Comment by Sir_Cmpwn | Hacker News Reader

Parent Full thread

HTTP is not designed for persistent connections (WebSockets were hacked on later, but JMAP does not use them anyway). This turns a push protocol into a poll protocol, which is a stark regression. It's also got all sorts of other crap built-in which is really not necessary for this use-case but which an intrepid implementer will have to deal with regardless.

JSON does not cope well with binary data, which is common in emails. It fails to elegantly deal with the various email encodings which exist in the wild. Consider as well that all JSON numbers are floating point numbers, despite the fact that floating point numbers provide absolutely nothing of value to this spec and in fact are more likely to introduce bugs than not. And embedded devices can't deal with floats quickly or elegantly, but still need to implement them if they want to use JSON-based protocols. And for what gain!

In short, JMAP reinvents the wheel but worse for the sake of making it easier for web developers to build web shit around email. This kind of stupid change for change's sake is an epidemic in the webshit scene. IMAP is warty but it's fine. Let it be. When your only hammer is JavaScript, everyone else's thumb looks like a nail.

Some remarks as a developer on Fastmail’s webmail, using JMAP daily but not having been much involved in its development:

IMAP really isn’t fine. If it was, mail providers wouldn’t have kept on making their own protocols/APIs for their mobile apps or their webmail. JMAP has various concrete benefits over IMAP. JMAP has been made by people that have been working with IMAP for decades, and have been actively involved in IMAP improvements over that time as well.

Pretty much all of your objections seem to me to either be misinformed, or addressed at https://jmap.io/. Strongly relevant are headings 2, 3, 4, 5, 6 and 8. (I could write more about individual issues, but I’ll just leave it at that.)

The main thing that is only kind-of addressed there is your objection to JSON number representation. JMAP does not state that JSON is the best thing out there; but rather that it is good enough. JMAP doesn’t use floating-point numbers, uses opaque strings for IDs (where many APIs people home-bake would use numbers), and expressly limits its integers to the −2⁵³+1 to 2⁵³−1 range where it does actually use integers. (And none of those places should ever be anywhere near that.)

I will preface this by admitting my only direct experience with IMAP as a protocol was with writing a few scripts to synchronise mail and having to fix bugs in the IMAP libraries I was using.

I must be missing something obvious -- how is IMAP a push protocol? All experiences with IMAP I've had are in the form of basic request-response flows.

Also, regarding JSON floats, most languages have the ability to give you errors if you encounter a float in a JSON field that is meant to be an integer. So it really shouldn't matter at all that JSON's floats are awful. I haven't looked, but does JMAP use floats?

I don't particularly love HTTP nor JSON, but if that's the only problem with JMAP then it's such a massive improvement over IMAP that we should have started switching to it yesterday.

Basically all IMAP servers found in the wild support IDLE:

https://tools.ietf.org/html/rfc2177

The problem with IDLE is that it requires one TCP connection per folder you want to watch. The better alternative is NOTIFY, but in my experience, it is not as widely supported by server and client implementations.

https://tools.ietf.org/html/rfc5465

But that is an extension so you cannot count on it being there. Thus a well-written client would need to handle servers that do not support IDLE. The X11 protocol is similar in that it has a lot of extensions that are theoretically optional but de facto required to deliver a good user experience. It makes implementing such protocols complex because you need to duplicate code paths, one for when the extension is present and one for when it isn't.

IDLE isn't sufficient which is why Lemonade was defined

https://tools.ietf.org/html/rfc5550

Except AOL and Yahoo.

The IDLE-command must be a push type of command? Which is used heavily by clients instead of polling.

As Microsoft does it with RPC is just a mess, they do a http call with 2billion bytes in request size, let the connection terminate at timeout and fire up a new call. I for one hope not that this is how JMAP solves it :(

If submit an app to Apple with IDLE support I would bet will be rejected. JMAP has PushSubscription which will play nice with Apple / Google push services.

JMAP doesn't solve it at all.

Sure it does. See https://jmap.io/#push-mechanism

JSON explicitly does not require floating point (https://tools.ietf.org/html/rfc8259#page-7). As a data-interchange format, it's not interested in most of the issues surrounding floating point numbers (operations, rounding, exception handling, etc), but it doesn't even use the interchange formats of floating point either.

I'm sure that as a practical matter, it will be a rare implementation that does not work by deserializing non-integer decimal numbers into floating point numbers, but the JMAP spec does not require this behavior, and I believe it uses no decimals.

I do hate this fashion of tunneling everything through HTTP, but then:

- Email protocols are all shit. The email ecosystem still didn't evolve into a format where we can safely transit binary data around. It is still risky to rely even on octet encodings, so there is very little binary data on emails anyway, it's mostly 7bit strings.

- The numbers in emails are nearly all strings. The very few integers are encoding content length and are not necessary with HTTP.

- We have much better odds of having message push that actually works everywhere with an HTTP protocol, mostly because many of our computers are configured to serve ads first, and ads use HTTP push.

That said, I do look forward into all the interesting problems HTTP will create for large emails.

> The email ecosystem still didn't evolve into a format where we can safely transit binary data around.

The only significant characters in email are carriage return, line feed, and period. There's also a line length limit in the SMTP protocol specification. Other than that, bytes sent during the DATA phase are sent unaltered.

Base64 encoding is meant to address this, but it results in a 33% overhead in attachment size. On the usenet side, people came up with an encoding scheme called yenc that actually only escapes those characters mentioned above and only as a 2 to 3% overhead over the original file size.

If you are talking about SMTP, servers may fail on receiving any character larger than 127, unless the client started the session with EHLO and the server announced either the 8BITMIME or the BINARYMIME extensions, where the first one only allows valid UTF8 stings, and the second requires a completely different mechanism that does not use DATA.

> Consider as well that all JSON numbers are floating point numbers, despite the fact that floating point numbers provide absolutely nothing of value to this spec

JMAP seems to not use any float, so implementers do not need floating point logic and can just reject JSON with floats.

Aren't most email attachments base64 encoded anyway? If so, JSON not supporting binary is not really a problem.

Converting to base64 generally increases the payload size. Being able to convert it that way is a workaround - not a solution to being able to transmit binary information.

Well yes, but all emails I've ever seen already have their binary attachments encoded as base64

I received two emails yesterday with application/octet-stream as part of the inner body, compliant with RFC2046 [0]. Having a Content-Transfer-Encoding of base64 is entirely optional.

It's the default for certain larger email hosts - that's the reason you see it frequently, but that in no way means that it is the _best_ option for a protocol that suggests it's solving the problems of the old one.

[0] https://tools.ietf.org/html/rfc2046#section-4.5.1

This isn’t your main point, but FWIW, there is a draft in the IETF for JMAP over websockets. It was approaching trivial to implement; I think our (Fastmail’s) main Cyrus developer wrote it from scratch over a weekend.

Reply on news.ycombinator.com