Ten Years of JMAP
fastmail.com
fastmail.com
It would be great that the two big email providers (Google and Microsoft) implemented and supported it.
It would make so easier and reliable to have a single client that works really well across personal and business email accounts, for example.
Hopefully, next year there will be JMAP support for Contacts. As RFC drafts, there are already JMAP specifications for Files and Calendar...
But Gmail doesn't have "folders". They have "labels". I wont go into all the details about how Gmail labels work that frustrate and annoy me, but the point is that IMAP clients cannot see labels, they can't replicate it on the client end, etc.
This caused a legitimate problem at work when the finance dept. wants to use this service that reads emails to automatically scan invoices into their system and whatnot. They wanted to be able to only have a subfolder automatically imported into the program, because too many of the emails were not relevant to get imported, creating more work to clear them out.
We were not able to achieve a solution with a simple folder for the users to move emails into to get them imported. Instead, we had to set up a second email account specifically for this importing service, and whenever finance wants something imported, they forward the email to that second account.
It's not a feature originally built for labelling like Gmail, so it has a few complexities ("Servers MAY permit the client to define new keywords") and enough clients (especially mobile) that don't support it - but they could just like Thunderbird or some webmail clients.
I've used it basically exactly like Gmail labels for years on TB on desktop.
I think the "NOT" slipped in there by accident.
But you have to ask on that scale, what would it bring them other than change if everything already works fine?
Having implemented JMAP and IMAP protocols myself, I haven’t encountered a need in either protocol to send binary data to the server for sync/search operations that would require the server to perform expensive parsing.
JMAP offers many improvements, such as a stable messageId for each message and a state mechanism that allows the server to be queried for changes since the last saved state. This avoids the need for numerous IMAP SELECT commands per folder to check the state using CONSTORE/QRESYNC.
If CONSTORE/QRESYNC aren’t supported by the client, it results in very costly chunked queries just to verify if message flags are still the same.
The same applies to SEARCH—if a user has many folders, it requires multiple network hops to SELECT and query each folder. With JMAP, this can be done in a single API call.
IMAP is absolutely terrible on unreliable connections. A connectionless protocol based on HTTP would do much better in those conditions.
IMAP is ugly but that is all well understood.
Note I've designed and built a complex messaging system that runs over HTTP over unreliable connections and run it in production for 15 years so I know what can and does go wrong.
Popcorn at the ready.
To some degree, that isn't true anymore as of last week but of course, you meant when you used it, not generally: https://www.fastmail.com/blog/offline-in-beta/
Useless to me. Seriously.
I write a lot of email on planes and do a lot of email management.
All that works on the default Apple stuff as shipped other than account management. Using IMAP and iCal etc.
And yet: at least as of a couple months ago, there isn’t actually any offline support! The Fastmail first party app does not work at all offline. Complete failure. Does not even try. Oh, and it has dramatically worse threading support than even Gmail or Outlook.
I don’t get it. POP3 is awful and kind of works offline. IMAP is old, clunky, and works fairly well offline. JMAP is supposedly the new hotness, but the paid first party experience will not even try to load offline. This is table stakes! Eudora could do this. Every version of Outlook ever could do this. Thunderbird can, and always could, do this. The usual command-line clients work offline if configured appropriately. Heck, Google went above and beyond and made Gmail’s webmail work quite well offline if you care to set it up, and I think they did a bunch of early work on service workers to make this possible. Fastmail, please stop pitching your fancy protocols until you can get your clients up to the state of the art as of twenty to thirty years ago.
edit: Huh, mobile offline support is in beta as of December 16. No way!
The app is ready for offline use, but I don't want to implement it without 'full text search.' I plan to use SQLite FTS5. My initial tests with search show that offline search is storage-expensive, which could be problematic for mobile devices. So, the edge cases have started to appear. But it's the most requested feature, and the most interesting/challenging to me :)
Or even worse, sometimes I look at some email in Mail and it’s there, and then hours later, while on bad network conditions, I look at it again and see “This email has not been fetched from the server. Download?”, which then just hangs.
I have a 1TB iPhone and a 2TB iPad. My main Gmail account is not large by any means - ~50k emails and 5GB space. Even though I have all this space on my iPhone/ipad available and willing to commit a bunch of it to email with all attachments and full text index, there seems to be no way to do it.
The Gmail app has a slider for “days of mail to sync”, but the max is 90 days, and not “All time” like one would expect. Anyways, good luck with Mailtemi!
Does not Thunderbird work with Gmail/Microsoft/etc? Does not Apple's Mail.app?
Targeted, focused conferences like Inbox Love, with 150ish or fewer attendees, are by far my favorite because you can actually get to know folks, ideas flow more easily, and everyone is focused on approximately the same thing. Much better than huge, multi-track conferences. We should host more of those as an industry.
How come there are no free of cost email providers that support JMAP?
I know a couple of people who run large email stacks (500k+ users) and this isn’t on their radar and there is no interest in it. There is a little disdain as they are comp sci people and don’t like JSON at all (I share that concern).
Email is probably the most conservative, reluctant to change set of protocols there is.
https://en.m.wikipedia.org/wiki/The_pot_calling_the_kettle_b...
At least no one tried to use YAML on the wire, so far anyway.
(Source: implemented both serverside and clientside IMAP.)
IMAP is basically a database query language and as such it works as it should.
. SELECT INBOX.Archive
* 1239 EXISTS
* 0 RECENT
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen $X-ME-Annot-2 $HasAttachment $IsNotification $IsMailingList $NotJunk $CanUnsubscribe)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen $X-ME-Annot-2 $HasAttachment $IsNotification $IsMailingList $NotJunk $CanUnsubscribe \*)] Ok
* OK [UNSEEN 1221] Ok
* OK [UIDVALIDITY 1108730350] Ok
* OK [UIDNEXT 2231] Ok
* OK [HIGHESTMODSEQ 40306873] Ok
* OK [MAILBOXID (210306ee-5833-456b-bede-6d04757128b3)] Ok
* OK [URLMECH INTERNAL] Ok
* OK [ANNOTATIONS 65536] Ok
. OK [READ-WRITE] Completed
. FETCH 1 MODSEQ
* OK [HIGHESTMODSEQ 40306873] CONDSTORE enabled by FETCH MODSEQ
* 1 FETCH (MODSEQ (39570709))
. OK Completed (0.000 sec)
. FETCH 2231:* (UID MODSEQ)
* 1239 FETCH (UID 2230 MODSEQ (40306872))
. OK Completed (0.001 sec)
And it is a pretty bad query language compared to something like SQL.
Yes. Regardless of the email protocol, third-party clients will always be horrible as long as corporations view email as their enterprise moat.
And it'd better stay that way, as it's also the most widely used set of standards for communication across borders, that people rely on about as much if not more than traditional telcos now.
(You should know why the common-sense comments are downvoted --- this is basically an advertisement article, and they don't like it when their $shiny_new_thing is actually shown to be worse than the existing solutions. A huge clue is when the adjective "modern" is thrown around like it's a good thing.)
They need to motivate people to use FastMail to boost JMAP adoption and they've failed to do that for a decade.
They post their 12 days of Christmas blog every year and never a new feature to be found.
If you can't add shiny features to your service with a new shiny protocol that promises to break down silos, then it doesn't stand a chance.
What will replace IMAP will be what changes email forever, and we've not seen that yet in an open protocol.
We should revisit Google / Apache Wave
Constantly reinventing things and piling on pointless features is one of the things I dislike most about Google, their chat solution is especially egregious for this. It only causes frustration and increases the time investment required to be proficient with the products.
I too like the feeling that they're stable. I guess they're just tactful with their additions.
Not invasive is a good way of putting it, I don't mind new features that are not invasive.
I'm suggesting that a new protocol needs hype for adoption and reinventing email needs new capabilities
The amount of times IMAP has resulted in duplicate emails when moving mail between folders for me is enough that I eventually gave up on attempting any kind of organization back when I was last using it.
Anyway, It has been some years since I last used IMAP. I had issues with Outlook, Thunderbird and Nylas N1 while connecting to Gmail, Exchange and Office365. It was a fairly small mailbox getting about 20k mail a year and keeping about half of that. Usually moving about 5k mail around at a time (end of year).
It got so bad that I had to get familiar enough with the IMAP protocol to make my own client program to handle deduplication overnight. Shortly after that I decided to just give up on trying to keep thinks organized.
I am in Australia and this was before the NBN, so obviously it was always on shitty connections.
JMAP seems like exactly the protocol I’d want to use to avoid much of the IMAP pain, but there’s almost no ecosystem.
If I wrote an E-Mail client for JMAP, I’d basically make it for Fastmail and for Fastmail only - and I couldn’t even do so without paying Fastmail. That’s untenable, I’d be 100x better off targeting the GMail API.
An IMO perfect way to jumpstart adoption would be an IMAP-to-JMAP proxy, which I don’t think currently exists (maybe due to basic deficiencies in IMAP, I’m not sure). It would allow people waiting to develop and use modern clients relatively easily.
Currently the best way to get non first party JMAP infrastructure seems to be to host Stalwart (https://stalw.art/) and do some trickery to forward your E-Mails there, which hasn’t been worth the effort to me so far.
Without more email clients supporting it, mail providers don’t have any incentive to support JMAP. Mozilla Thunderbird started looking at JMAP but hasn’t progressed on that all these years.
> In 10 years time, I hope to post about how Cyrus and JMAP have taken over the world
I believe the biggest hurdles are other email clients and providers not adopting it. The biggest threat is Microsoft, as usual, pushing its own protocols and client and using FUD to brainwash CIOs into believing that any protocol outside its own is a major security threat that just cannot be handled.
If either Apple or Google could be convinced to implement and support JMAP, this could take off a lot faster.
I like Fastmail and use it because it provides a service, and does so well (which includes support you can actually reach out to and talk to a human who will actually help you). I also like that my identity and daily e-mail experience is in no way hard-linked to them. I have my own domain, and I use Thunderbird and K9-Mail (now also Thunderbird).
I also like that I can log onto to their website and do things there if needed. This is mostly managing masked e-mail addresses and the occasional check on my subscription settings.
And if you're going to fund development, also put it in Dovecot.
* https://en.wikipedia.org/wiki/Dovecot_(software)
It does seem to be in Cyrus, but I'm not sure what the marketshare of that software is nowadays:
I wonder what the mobile app adds beyond what the platform-agnostic webapp already does.
They cited IMAP's complexity, high resource use,
JMAP is implemented using JSON APIs over HTTP
They complain about complexity, then add another two layers of complexity in their own protocol?
I think POP3 is the simplest standard, and have also written a basic IMAP client. Parsing IMAP isn't as easy as a binary protocol, but it's definitely not at the level of HTTP JSON bloat that seems to have infected all "modern" protocol designers. I can use POP3 reasonably easily from a netcat (and have done so many times in the past), and IMAP is a little harder but doable. I don't expect that to be doable for JMAP which is text-based like the other HTTP JSON bloatocols, but unlike the earlier text-based standards like SMTP POP3 IMAP IRC MSNP etc., it seems to have all the disadvantages of a text-based protocol but none of the advantages.
JSON is only excusable if your protocol is designed for webmail clients. Otherwise this is a case of the emperor having no clothes.