Hacker News Reader

Comment by jeffbee | Hacker News Reader

The fact that JMAP has ~zero adoption outside Fastmail itself contradicts the "much needed" part of this title.

The problem is that email has been taken over by a handful of providers, all of which have "growth & engagement" as their business model and would rather not implement any open protocol at all.

IMAP remains supported for backwards-compatibility, but I'm sure its shortcomings are seen as a feature to push users towards the official clients so there is no incentive to implement another open protocol to address them.

This is a problem, but the other problem is the tools to implement the service. You can use Cyrus, Apache James, or Stalwart, and that's it. These tools do not integrate with anything; in the case of Stalwart (the only JMAP server not in "experimental" status) you can't just install a JMAP server attached to an existing email service; you have to hand it complete control of everything, even unto the on-disk storage of mail. This is a complete non-starter for any email service outside of hobby/personal hosting.

In other words, even if someone wanted to make JMAP available to their users, they'd have to tear the whole thing down and rebuild it in JMAP's image, and the tools available to do so are not fully-featured or integratable enough to do so.

> These tools do not integrate with anything; in the case of Stalwart (the only JMAP server not in "experimental" status) you can't just install a JMAP server attached to an existing email service; you have to hand it complete control of everything, even unto the on-disk storage of mail.

This point you mention is being improved. The next release of Stalwart JMAP (expected in one or two months) will delegate user management to either a SQL database or an LDAP directory. Messages will be stored in either Maildir or MinIO/S3. And all other information will be in either SQLite or FoundationDB. All these features were already implemented except MinIO/S3. Development progress can be tracked at https://github.com/stalwartlabs/mail-server/tree/main/crates...

Cyrus is what fastmail use for their own mail servers. And even the fastmail web ui uses jmap.

I don’t know where it says jmap support is experimental, but I think it’s pretty stable and well supported in practice. The main maintainers of the project depend on it utterly.

They don't use the word "experimental" any more but they call it a "work in progress": https://www.cyrusimap.org/imap/developer/jmap.html bottom of the page.

The file storage approach is unfortunately super common and makes a bunch of really basic things difficult and slow. Need to phase it out more heavily, especially from new implementations.

Telling people who have decades of perfectly serviceable email solutions that they need to 'phase out' what works is a great example of an unconvincing argument. What you say might be true from a developer's perspective but maildir is a battle-tested, accessible, well-understood and interoperable storage format. Bundling everything up behind some bespoke SQL schema might ease performance hurdles but it sucks for everything else.

Does it work though? Nobody runs such systems at scale, for very good reasons, because maildir just sucks in so many ways. Horses have also been battle-tested, they're well-understood, but they don't transport tonnes of cargo and getting kicked in your face because they don't like you isn't pleasant.

So if it takes a custom schema in a (No)SQL database, so be it, it's simply so much more performant, scalable, reliable and usable. It's simply idiotic how much maildir-based software starts "dying" at around 100000 emails in a folder, in 2023!

Yep. If you go and register a new gmail account today, there will likely be no option to enable IMAP access for that account, altogether.

That appears to be incorrect. https://support.google.com/mail/answer/7126229

Why would you post a baseless conspiracy theory that is easily disproven by anyone in less than a minute?

FYI, the above is true for "child accounts", as my son painfully discovered the other day, after his school shoved Gmail down his throat. No IMAP in the settings, or anywhere else, for the next generation of users, not until they are 16+ in my country.

What makes you think this is for youth accounts, and is not simply the preference of the workspace administrator for your school?

Why would you need to enable IMAP ? It works out of the box. Every non-android platform uses IMAP to access your account (k9 mail, outlook, thunderbird, the iOS/mac email client, etc).

You do have to go into the gmail settings to enable IMAP, FWIW.

As a former K9 Mail developer I will say that the Google extensions to IMAP for Labels and OAUTH (neither of which are standards) are a pain in the backside that no email platform would have dealt with were it not for the fact they are huge.

(I did a spike development for OAuth and a separate investigation into Labels when I had some free time).

So they are definitely not nice IMAP citizens.

Oh and then there's the issues with Labels that their own documentation apparently gets wrong...

To be fair here, labels are a great feature and the default limitation of a letter existing in only a single folder is a dumb remnant of ancient times.

If it's still not standardized then the pitchforks are aimed in the wrong direction.

Are you sure?

Because, IIRC, IMAP access must be explicitly enabled by going to Settings -> Forwarding and POP/IMAP -> IMAP access: (o) Enable IMAP. And I think it is not enabled by default.

Case in point: MS making it more and more annoying to use IMAP.

Hell, now you need to create oauth2 app just to login from non-approved client, as they recently forced OAUTH2 auth on IMAP clients. Sure thunderbird works but most lesser known clients require a lot of fuckery to make work (and possibly admin permissions for domain, not sure).

https://github.com/simonrob/email-oauth2-proxy

"Just works" I run it with mbsync at the command line.

You do need a client id and some interaction with your O365 admin.

This looks promising, thanks.

I did the song and dance for claws-mail and one day it just... stopped working ;/

I would be migrating away (to anything) if my email provider pulled that on me.

It's entirely idiotic history.

We had our own e-mail, but we had some problems that were generally caused by management wanting @example1.com and @example2.com to be entirely equivalent, not just an alias, but still land on the same account.

which was fine for the email, as it can just have custom rules, but any related services like calendars bugged, as it saw different domains as different users. We tried to hack around it but it was still buggy in places.

Boss finally got pissed off and decided to migrate to O365, all while IT was screaming nooo. "Because our clients use it too, it will be easier to collaborate"

...and it turned out that what he wanted isn't possible in O365 *fucking anyway* so all e-mails are under @example1.com. And we replaced some weird quirks with many more quirks we can't even hope to fix. We waste more time dealing with this shit than we did on running our own mail servers. Which we didn't like, coz running mail servers sucks, but O365 sucks harder.

My university uses MS for their services, and they stubbornly refuse to let me use IMAP (not even with OAuth). I have to either use GNOME Evolution for Micro$oft exchange protocol or a proprietary app on my phone, which enforces bullshit control on a device I paid for.

> which enforces bullshit control on a device I paid for

That's basically the raison d'etre. You, presumably a student, are getting caught in the crossfire of regulatory compliance and they don't want to maintain two email systems.

Source: Used to be the one implementing those controls. We didn't like them either, I promise.

Effectively, MS has bulldozed companies that use Microsoft365 into using its own proprietary protocols and applications (Outlook). It scares companies into not enabling IMAP (even with OAuth2).

It really really hasn't been. Email is just an enormously massive ecosystem with tremendous amount of variance that anything new will take a long time to adopt and a lot of convincing and proving.

Just look at how far the deployment of something as essential as DMARC is, it's sad. That's not some big vendor's fault, it's all the small legacy cruft nobody wants to clean out as it might break things.

There is no incentive for existing email providers to adopt new standard. They would never adopt something new & improved unless it's groundbreaking.

You can cry conspiracy all you want, but the fact is that even Thunderbird, the flagship open-source email client, does not support JMAP.

To anyone interested in bringing JMAP support to Thunderbird, the Bugzilla bug is here:

- Add support for new JMAP protocol: https://bugzilla.mozilla.org/show_bug.cgi?id=1322991

There's also Ltt.rs, a free and open source JMAP email client for Android:

- Ltt.rs source code: https://codeberg.org/iNPUTmice/lttrs-android

- Ltt.rs on F-Droid: https://f-droid.org/en/packages/rs.ltt.android/

Can one donate specifically to Thunderbird to implement JMAP?

I don't think so, but you can ask Thunderbird if they are willing to earmark your donation for JMAP specifically:

> Who can I email directly with questions about giving?

> If you have a question about giving to Thunderbird, please contact us via this form. We will do our best to follow-up with you as soon as we can. If you need technical support, please head over to Thunderbird Support for assistance.

https://www.thunderbird.net/en-US/donate/

You can also try contacting the developer who offered to work on JMAP integration for pay:

> I can't commit to a project of this size however since I work freelancing and my availability is fluid. I am open to being sponsored/contracted in order to do this as part of my job.

https://bugzilla.mozilla.org/show_bug.cgi?id=1322991#c23

What would be the benefit to using JMAP in thunderbird?

JMAP main benefit is that it allows a mail client to to things efficiently in JMAP that are extremely difficult to do in IMAP or take a long time to do. Merely adding JMAP support in Thunderbird will not add any benefit.

The benefit will come if Thunderbird is refactored internally to make use of the nice properties of JMAP, but then there will be a problem with IMAP compatibility which must be kept.

JMAP is nice for new mail clients that can move away from a folder-centric paradigm, but if you already managed to use IMAP then it's less useful.

I tried to implement this for quite a while but the ecosystem situation both server and client seemed to be a status of "Weekend Project" rather than a robust implementation. Is there commitment to create a reference implementation etc.

The issue is that there's a single monopoly provider who very much likes forcing all of the client apps to support their proprietary API. When 70% of users are Gmail, and Gmail has a vested interest in not supporting JMAP even though it's better, it's a hard sell for other client apps to invest effort.

The solution, of course, is monopoly regulation. AOL Instant Messenger was forced to interoperate and embrace standards. If our legislators had the actual courage to do their jobs in this day and age, Google would be legally required to shift Gmail features to using open standards like JMAP.

Mimestream[1] is a new email client for Mac. It’s currently gmail only but JMAP support is on their roadmap.

https://portal.productboard.com/mimestream/1-mimestream-road...

If anyone else would like to see it supported consider upvoting on their roadmap.

[1]: https://mimestream.com/

Unmet needs are legion.

Reply on news.ycombinator.com