Hacker News Reader

Comment by LeoPanthera | Hacker News Reader

This might be a naive question, but I don't know any better: Why can't your mailbox be represented on a server as a directory, perhaps like Maildir format, and then synchronised to your device using a directory sync tool, such as rsync?

Changes, or online access, could be done using any standard remote file access method, such as sftp.

Are there downsides to this approach?

View on HN

You still need an index of your mail for searches and extra metadata for tags, threading, unread status etc. So you need a protocol for requesting this search and a format what how the client can use the result of that search to access the matching emails.

You need to be able to push email status changes out to connected devices, I don't want an alert for an email on phone if I've already read it on my laptop.

You need decisions around how attachments are stored. Most of the time you'll want to store the attachments separately from the text content of the email so clients don't have to download the attachments to access the text, so you'll need to decide on how this location would be communicated to a client.

Plain files are rather bad for multi-user access, if you have multiple users/devices all accessing the same email they're going to need some protocol for synchronization so you don't get data corruption or lost changes.

It would be interesting to think about email and sync using git on the backend.

Perhaps something like having each blob refer to an email message. Each tree would refer to a folder. Each commit would be made whenever an email is added, deleted, or moved to another folder (though it would not have a reference to a parent). For indexing, you could refer to each blob with a tag that had a name that corresponds to a UID. If an email is deleted, then the corresponding tag would be deleted as well.

Git's garbage collection would eventually remove references to orphaned objects and the transfer protocols would only push (or fetch) objects the other repository doesn't have.

I don't know how it would scale in terms of performance though.

> You need to be able to push email status changes out to connected devices, I don't want an alert for an email on phone if I've already read it on my laptop.

This. This sucks so much. Even gmail can't automatically dismiss a notification when I read a mail on the desktop!

What mobile client are you using? I've found that Gmail on Android is pretty good about dismissing notifications for messages that have been marked as read elsewhere.

Yes. Combine your approach with a new device, a mailbox size of several gigabytes and EDGE/2G network connection.

IMAP/JMAP will work fine in those conditions, as a client can simply fetch the list of folders and a few email headers, and then lazily fetch email bodies on demand. Updating the read status is pretty cheap and does not require a full sync between the device and server, and so on.

This will be way too slow, and is not at all feasible on a device like a mobile phone where you want only the recent email cached locally, but want to be able to stream on demand the rest with minimal latency (so it appears to the user as though it's all loaded locally under ideal network conditions). The server needs to be able to sort the messages, calculate threading, unread counts etc.

rsync is apparently extremely inefficient in a number of common cases, such as simply renaming a folder (completing synchronizing all the files in the folder again).

Also, some IMAP servers store message flags in the filename (when using Maildir). This means that a message that goes from unread to read, it would appear to rsync that one file was deleted and another created (as the filename on disk has changed); the "newly created" file (which could be many MBs in size, depending on its contents/attachments) would then have to be completely synchronized again.

I didn't realize just how inefficient it was until a few days ago when I read an article [0] comparing rsync against ZFS replication (even if you're not familiar with ZFS, you'll understand).

[0]: http://arstechnica.com/information-technology/2015/12/rsync-...

> rsync is apparently extremely inefficient in a number of common cases, such as simply renaming a folder (completing synchronizing all the files in the folder again).

If you run it the normal way this is (sadly) true.

However you can use the `--fuzzy` flag to make the behaviour a little more clever in handling this specific case.

That's what I'm doing. My maildir is just synced from the server with Syncthing.

How would you make this work on iOS/android?

Reply on news.ycombinator.com