Hacker News Reader

Comment by ar7hur | Hacker News Reader

I've used the Nylas API for 9 months for my side project iOS email client. It works very well and the team is super responsive. Documentation is clear and complete. It makes it super easy to build any email client.

Eventually though, I decided to stop using it because I'm not confortable with having all my messages stored by "one more" entity in the cloud. Back to good old IMAP with all the pain it brings.

I'm often wondering if such an API could work in a full end to end encrypted mode, i.e. messages would be encrypted as they arrive before they are stored on the server, and only the client would have the key to decrypt them.

View on HN

Engineer from Nylas here: End-to-end encryption is definitely on the roadmap. In the meantime the Sync Engine service that we're currently hosting for the sake of convenience is fully open source. You can run everything yourself on your own machines and still have the modern RESTful API that Nylas exposes. No more IMAP!!

There's documentation about how to get that setup with N1 here: https://github.com/nylas/N1/blob/master/CONTRIBUTING.md

There's much more on the Sync Engine page here: https://github.com/nylas/sync-engine

> No more IMAP!!

Why is that a big deal? I've worked with IMAP, and it's not that bad.

> I've worked with IMAP, and it's not that bad.

I don't believe you.

Every email account provider is different in horrible, horrible ways.

It really is. It's ancient, stateful, no one is compliant (everyone has their own extensions), doesn't provide easy syncing, and the guarantees for id consistency, well, aren't there. You can't simply say 'this was the last delta I saw, give me all the updates for my entire user'. That's why email clients spend so long 'syncing' across all the mailboxes.

I've recently started working on an application using email and was considering using Nylas but have hesitated for the same reasons as you. But IMAP is such a pain that I may go ahead anyway. Have you considered running their sync engine yourself?

> But IMAP is such a pain

Any chance you could tell us what the pain is that you're experiencing with IMAP? I don't think I've ever had any problems with it, aside from offline (but for that there's POP) so I'm interested to hear your experience.

There's two things primarily that burned me a few years ago when I was working on an MUA:

1. Message Identifiers/Primary Keys - if I recall correctly, IMAP doesn't have a way to definitively map a message and a unique identifier to the message in a way that is guaranteed to persist long-term.

2. Not exactly IMAP, but malformed messages. When I was working on this, I was using my GMail corpus, which goes back to about 2006 or so. Somewhere around 30-40,000 messages. Turns out it was a ridiculously good test case for parser robustness. Messages that claim to be ASCII-7bit, but have extended characters. Messages that claim to be UTF-8 but have invalid codepoints. Messages with malformed MIME headers. Messages with a text/plain section with garbage in it, while having a nicely formatted text/html section. Messages with embedded CSS that tries its hardest to blow yup on-screen display.

I think that through the duration of the project, I only had to submit 2 PRs to the upstream net::imap ruby library, but damn were there a lot of heuristics in place to try to successfully grab an email from IMAP and display it on screen.

And that's before you even consider building Microsoft Exchange ActiveSync...

My app does a lot of folder/label management on the user's behalf, and some of the issue I've run into are:

* UID MOVE not always being supported (meaning you need to copy and expunge), and weirdness around gmail's IMAP extensions. This could be abstracted away with a good high level library, but I haven't found any for Ruby that covers all the cases properly.

* Gmail offers OAuth, but for regular IMAP, needing to store user's email and password and the security risks that come with that.

* Needing to know when new messages arrive, so either polling regularly, or keeping a bunch of connections open and using IDLE.

Nothing unsurmountable, but using a service like Nylas means I can spend more time developing features instead.

Well, the Nylas engine doesn’t provide any authentication by default. It just assumes if you enter the username correctly, the account is yours.

Which makes it hard to self-host without building a different auth system around it.

What I mean is, Nylas takes care of asking for and storing the user's credentials to their email account(s), all you need to store is the token that accesses the Nylas API.

Reply on news.ycombinator.com