Nylas N1 – extensible open-source mail client
nylas.com
nylas.com
> See the README for full instructions on setting up the sync engine, API, and N1 on your local machine.
Sadly, there is no instructions in the linked document yet.
I wonder, can the sync engine and whatever else be bundled with a client, so it'd be a self-sufficient standalone piece of software?
Sorry-- launch day craziness :) More docs to follow soon!
But I can see plenty of reasons the design was chosen. If you are bringing in lots of mail accounts, having it running as a daemon isn't necessarily a bad way to go. A sync program doesn't need to be associated with a tty, is better off run in the background in its own process, is arguably more extensible in design terms, doesn't necessarily take up any more memory than a regular executable, allows for multiple UIs (e.g. terminal program, Gtk+ GUI, WxWindows GUI), seperates processing from the UI, lots and lots of reasons.
Maybe the developers can step in here with a response?
You can make a UI to turn off the service or use the command prompt, if that's the concern.
If you can give me a valid technical reason running the sync part of the program as a daemon is bad, I'm definitely all ears though.
Seems like they've addressed at least one of your concerns, and it might actually turn out to be very simple to set up locally, so here's the privacy trade-off for not using it in your case: privacy > value || privacy > work_of_local_install
You are already doing this unless you run your own mailserver, or if any of your inbound/outbound emails aren't GPG encrypted (or encrypted during connections with other mail servers using TLS).
Something has to receive email for you when your computer is not running, unless you simply rely on a local mailserver, updating your MX and A records whenever your IP changes, and let other mail servers queue your email until your local desktop is back online to accept them.
[1] http://jmap.io/
"JMAP is a transport-agnostic, stateless JSON-based API for synchronising a mail client with a mail server. It is intended as a replacement for IMAP. The specification is based on the API currently used by FastMail’s web app. It aims to be compatible with the IMAP data model, so that it can be easily implemented on a server that currently supports IMAP, but also allows for reduced data usage and more efficient synchronisation, bundling of requests for latency mitigation and is generally much easier to work with than IMAP."
They're merely implementing their own means to access email in convenient and modern way. Say, IMAP has no labels[1] or contacts[2] (and many other things) - there are proprietary extensions, MS Exchange being the most famous one, but they're... proprietary. And given that proprietary implementations differ, it makes sense to create an unifying wrapper.
And JS apps and web services are just an approach that's trendy today. Personally, I don't like it either, but can't blame for going with the flow. Especially since it's a business (and going against the flow has costs, both in development and adoption), not a private project where one can do things in a way that just feels right, ignoring anything else.
I believe it would absolutely make sense to bundle the "sync engine" within the mail client, though, to make it standalone. But I understand it's done this way just because there was only the engine (as a service, not a library) at the beginning.
____
Everything you ever wanted to know about Exchange [1] (but were afraid to ask). I personally use SOGo [2] as a Exchange ActiveSync provider but I know OpenChange is a project to implement the entire Exchange stack. So I would not consider it proprietary.
[1] https://msdn.microsoft.com/en-us/library/cc307725%28v=exchg....
Sounds like it's time for a new protocol, then. Is anything in the works?
http://serverfault.com/questions/115769/which-imap-flags-are...
I'm not entirely sure what's "happened with email in 10 years"? Search-defined virtual folders?
Mark Crispin was actually pretty-damned prescient when it came to IMAP: he believed that client operations should be offloaded to a server that would have much more disk space and CPU than the client, which is exactly the model everyone reaches for today, and thereby designed a thin client protocol that had asynchronous updates baked into its core. A few years ago, he passed away: here is what I wrote at the time about his mission for IMAP and history on the project.
https://news.ycombinator.com/item?id=4826113
Really, the only thing in IMAP to actively dislike is how the thin client view requires the server to have a "dense" (here I mean the mathematics term, not from the technical spec) numbering of messages the client is modeling in its view (this is the reason opening a large mailbox via Zimbra takes forever, and why many IMAP servers have dumb global locks), but I have come up with some techniques that should allow me to handle that with minimal overhead. Otherwise, with some of the modern extensions in place (QRESYNC, for example), IMAP is pretty damned awesome.
> "Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius -- and a lot of courage -- to move in the opposite direction."
Though I think I get why this email client and Android's Outlook [1] will hoover your emails to some other server - the server may have more processing power than your device. And/or use an RDBMS to index the emails which is better suited for a server than a smartphone. And that's fine - but the email client should be able to do this on it's own without the need for a server (and explicitly show a warning/message allowing you to opt-in to using the third party service). It will be slower indexing emails on your phone but that is the price you will pay.
And to be fair - I've tried almost every email client I can get my hands on: Evolution, Foxmail, Pegasus, Thunderbird, Outlook, ElementaryOS. To date no one has gotten calendar, contacts, and email better integrated than Outlook. Trust me I REALLY REALLY REALLY don't want to use Outlook - but I've yet to use an open source alternative where in 5 clicks or less I am syncing all those items previously mentioned. Thunderbird comes in second place - but I have to install 2 different extensions for contact and calendar syncing. The contact syncing has been hit/miss and the calendar syncing was stable for the most part. I should note that I don't use Exchange but I use iredadmin with SOGo.
I think SyncML was supposed to solve the whole syncing problem - but never really took off.
[1] https://play.google.com/store/apps/details?id=com.microsoft....
But why is that a problem? It's open source, someone should just package everything up into one nice downloadable desktop application.
Front-end is gorgeous. Using it to try out some ideas re: email encryption.
We've added (and are continuing to add) more documentation about how to get the open source sync engine setup.
It uses no effective authentication and everyone can just login to your accounts (due to /accounts and /n, the endpoints displaying the auth codes, being public)
Eventually though, I decided to stop using it because I'm not confortable with having all my messages stored by "one more" entity in the cloud. Back to good old IMAP with all the pain it brings.
I'm often wondering if such an API could work in a full end to end encrypted mode, i.e. messages would be encrypted as they arrive before they are stored on the server, and only the client would have the key to decrypt them.
Any chance you could tell us what the pain is that you're experiencing with IMAP? I don't think I've ever had any problems with it, aside from offline (but for that there's POP) so I'm interested to hear your experience.
1. Message Identifiers/Primary Keys - if I recall correctly, IMAP doesn't have a way to definitively map a message and a unique identifier to the message in a way that is guaranteed to persist long-term.
2. Not exactly IMAP, but malformed messages. When I was working on this, I was using my GMail corpus, which goes back to about 2006 or so. Somewhere around 30-40,000 messages. Turns out it was a ridiculously good test case for parser robustness. Messages that claim to be ASCII-7bit, but have extended characters. Messages that claim to be UTF-8 but have invalid codepoints. Messages with malformed MIME headers. Messages with a text/plain section with garbage in it, while having a nicely formatted text/html section. Messages with embedded CSS that tries its hardest to blow yup on-screen display.
I think that through the duration of the project, I only had to submit 2 PRs to the upstream net::imap ruby library, but damn were there a lot of heuristics in place to try to successfully grab an email from IMAP and display it on screen.
* UID MOVE not always being supported (meaning you need to copy and expunge), and weirdness around gmail's IMAP extensions. This could be abstracted away with a good high level library, but I haven't found any for Ruby that covers all the cases properly.
* Gmail offers OAuth, but for regular IMAP, needing to store user's email and password and the security risks that come with that.
* Needing to know when new messages arrive, so either polling regularly, or keeping a bunch of connections open and using IDLE.
Nothing unsurmountable, but using a service like Nylas means I can spend more time developing features instead.
Which makes it hard to self-host without building a different auth system around it.
There's documentation about how to get that setup with N1 here: https://github.com/nylas/N1/blob/master/CONTRIBUTING.md
There's much more on the Sync Engine page here: https://github.com/nylas/sync-engine
Why is that a big deal? I've worked with IMAP, and it's not that bad.
I don't believe you.
Every email account provider is different in horrible, horrible ways.
A few Nylanauts will likely hang out in this thread today. Would love to hear what folks think! :)
We've spent almost 2 years building the sync technology, so it's very stable and flexible. You can check out the IMAP code here: https://github.com/nylas/sync-engine
If, on the other hand, this makes use of some Nylas-hosted server, I'm a bit bummed out, since this would mean all my data is transferred through Nylas servers.
Which is it? Or should I just bite the bullet and read the source?
We're working on putting together a better doc that outlines how to set up the entire stack locally. Ideally this could even be a Docker container.
And you should always read the source! ;)
Can it use Gmail's classification of e-mails as "important", "not important", "updates", or "promotional"?
Unfortunately Gmail doesn't expose updates/forms/promotional via IMAP, so we can't show those. But they probably wouldn't be that hard to add in a plugin... ;)
Then your label will appear in your desktop client. Kind of a hack but I did this for thunderbird and it works.
As for making a plugin that handles this automatically - I'm not sure it's even possible, but I'd love to be proven wrong!
On second thought, the initial setup doesn't really need a plugin so much as the way categories are displayed. Especially cool would be to have them as tabs like in the web interface: once you get used to them, you find them sorely missing in desktop clients. I always find it weird to have my personal, social, and "promotions" emails brutally mixed together.
The FAQ sent me to the README for a completely-local installation instructions that I would like to try out, but they're not there. Where can I find them?
Thanks :)
Also, want to jump in our community Slack channel and chat with us? Getting a full environment set up can sometimes be a bit tricky and we can walk you through it. :) http://slack-invite.nylas.com/
The only issues I'm seeing are around HTML email rendering: https://www.evernote.com/l/AAI3x17h1EJCnZcbJV19q5AUB6G_vgOzI...
Can you create a GitHub issue with the email HTML? Or if you feel comfortable, forward it to support@nylas.com.
Thanks!
1. ability to snooze emails
2. rich handling of things like calendar invites, flight reminders, attachments, etc.
3. Vim keybindings for message composition
From other comments, it sounds like you're working on (2). How about (1) and (3)? I took a look at some of the documentation for plugins. Do these sound like features that could be easily added through plugins?
For example, Atom has a package for Vim keybindings [1]. How far from has N1 diverged from "dropping Atom packages in"?
1. Snoozing emails would be great. You could totally do this client-side in a plugin using our current developer APIs, by moving snoozed threads to a label/folder, and then moving them back after a prescribed time. Might be a fun project! Feel free to ping us on our community slack channel if you want to try.
3. N1 uses the exact same package structure as Atom, but we're using the CommandRegistry less, and introducing our own UI layer concepts like the ComponentRegistry (for dynamically injecting React elements.) There are probably a handful of Atom packages you could load into N1, but probably only well-contained ones with very minimal UI. Our composer unfortunately doesn't share any code with Atom's text view - we needed to do rich HTML editing, so we wrote our own composer based on `contenteditable`.
Vim keybindings for the composer are also a great idea. There's actually an API for creating `DraftStoreExtensions` (https://nylas.com/N1/docs/DraftStoreExtensions.html), and if you implement `onInput` in your extension you can probably re-create vim keyboard handling. Unfortunately, that's a bit low-level: right now, the composer is not as easy to extend as Atom's editor. We've put a lot more effort into extensibility of the thread / message panels and views.
Enjoy! Feel free to reach out on the community slack channel if you have any other questions.
Why should I download this client? What does it benefit me, that mail.app, outlook, mailbox or Thunderbird can't give me?
Since it provides up to 10 accounts for free, it looks like that end-users (for whom 10 accounts are more then enough) are not supposed to be the paying customers, but rather other companies that want to integrate with Gmail, etc.
But... does this mean that these companies are supposed to encourage the user to connect their e-mail accounts to the company's Nylas platform account, thus giving Nylas all the user's e-mails and weakening e-mail privacy even more?
Or is the business model something else...?
N1 is a new platform, and fits within our free usage for developer accounts. Paid features (potentially via paid plugins) are coming later. For now, we're focused on ironing out the UI/UX bugs, scaling onboarding system, and opening the codebase to developers.
We're a product company, not an advertising company. We have no plans to sell user data or build the kind of targeting/tracking systems used by Facebook and Google. Our goal is to build rich new experiences and tools on top of email that empower developers and end-users.
Privacy and security are obviously hugely important to us, and we know that it's our job to earn your trust every day. It's one of the reasons we've open sourced nearly all of our codebase. We make money to continue developing this product.
Does that make sense? Happy to dive in more if you'd like, or you can ping me directly: mg@nylas.com
I'd rather have all my data myself so I could pick great features from anyone who makes them. Nylas sounds like a part of the puzzle to make that happen. I wish you guys the best of luck.
In terms of privacy, the experience is good. I must warn, though, it's quite unpleasant in terms of dancing pigs and bunnies[1]. You won't get anything fancy. The age of protocol development had passed long ago and mail protocols (IMAP) aren't evolving anymore. Almost all the modern stuff is proprietary to services and their APIs and clients.
This limitation doesn't apply to mail filtering and automated parsing, though. Mail filters and processing for self-hosted email is way more powerful one can normally get with third-party services.
It happens that I'm right in the middle of configuring my own server. I've got only postfix and dovecot for now, nothing fancy yet.
If this is not powerful enough, you can make sieve plugin invoke external program and pass it MIME-encoded email message for further processing. This way your server can, for example, add events to your calendar software or do the accounting based on receipt emails (and I forward my SMS to my IMAP server too)
Before Sieve I've used tools like procmail (http://tldp.org/LDP/LG/issue14/procmail.html), although I'm currently only using Dovecot/Sieve setup for my own mail. Some servers I've configured, for example a DIY SMS gateway at the company I've worked for, use just postfix+procmail (+ custom bash and Python scripts).
As for the spam filtering, I've had a good experience with rspamd (https://rspamd.com/), which is light on resources, integrates with Postfix quite nicely, usable out-of-box and is scriptable with Lua.
Hope those few pointers would help.
Thanks a lot for recommending Sieve, I really mean this. There isn't even a single mention of it on the myriad of the half-baked 'how to setup isp-mail for personal use' blogspam I keep stumbling into when googling issues.
It looks like a great tool. Now I can probably have 'n' junk mailboxes with varying personal filters (always wanted this).
Our ultimate goal is to finally give you and others like you an outlet and set of tools to build the next generation of awesome features that currently only Google and a few others can do.
Then, Roundcube became the standard.
Their dominance is being eroded by Mailpile, RainLoop, and Nylas.
Around the edges, we have Peps[1], Mailr[2] & Kite[3] which may all someday take off.
Zimbra and the well known groupware vendors (OnlyOffice, Horde, Citadel, Kolab) are competing in more-or-less the same space.
Probably dozens of other projects that are on the same level that I have never even heard of. I would love some sort of comparison, even a biased one written by the respective teams. Anyone have useful links?
[1] https://github.com/MLstate/PEPS [2] http://pusto.org/en/mailr/ [3] http://khamidou.github.io/kite/
I find it hard to follow conversations without some form of visual indicator of who replied to what within a thread.
We may add something like that soon. I agree that it's superior in some situations. But most folks are familiar with Gmail, so we opted for linear threading for the first release.
Would love to see a plugin (or several) that explore new thread UIs! Here are a few you can look at to get started: https://nylas.com/n1/examples
[1] http://jmap.io/
---> Cleaning apm via `apm clean`
dyld: lazy symbol binding failed: Symbol not found: _node_module_register Referenced from: /Users/karl/tmp/N1/apm/node_modules/atom-package-manager/node_modules/keytar/build/Release/keytar.node Expected in: dynamic lookup
dyld: Symbol not found: _node_module_register Referenced from: /Users/karl/tmp/N1/apm/node_modules/atom-package-manager/node_modules/keytar/build/Release/keytar.node Expected in: dynamic lookup
/Users/karl/tmp/N1/apm/node_modules/atom-package-manager/bin/apm: line 28: 27072 Trace/BPT trap: 5 "$binDir/$nodeBin" --harmony_collections "$binDir/../lib/cli.js" "$@"
Thanks for checking out N1! I work on the client. Unfortunately `keytar` is a bit of a nasty module. Try using `nvm` to run script/bootstrap with Node 10.x. I think this might be the issue you're seeing: https://github.com/atom/apm/issues/195.
If you run into anything else, feel free to ping us on the Nylas Community slack channel (http://slack-invite.nylas.com/)
Most of our complexity is in the sync later, so we spend more time tuning our Python sync server. For example, we wrote a custom statistical profiler: https://nylas.com/blog/performance
I'm currently in doubt to move to either Mailpile or your platform on my personal server. However, PostgreSQL would be a big plus for your platform, since I don't have any experience with MySQL in recent years, and I rather do not run it on my VPS just for mail.
It's not something we're likely to support officially in the near future since we don't run PostgreSQL and don't have the ability to test it well with production loads, but it'd be neat to see a working community fork.
Relatedly, if any experienced devops/sre folks are looking for a new job, please ping me. ;) mg@nylas.com
Hopefully invites will go out over the next few days. Thanks for your patience.
I can't fully test it yet since I don't have an invite. It starts up, though.
Most frustrating is that search doesn't work at all.
I have multiple accounts, but if I didn't I would still prefer the gmail web interface to any thick client app I've used (so far).