Hacker News Reader

Comment by jsilence | Hacker News Reader

I really wish TB would store mails in Maildir by default and operate on that storage according to the standard so that the local mailstore becomes compatible with other mail clients.

Right now one can manually switch the storage to maildir, but it is not safe to use that storage with other maildir compatible software.

View on HN

Maildir only suggests the safe delivery mechanism that gets around lack of atomicity guarantees in filesystems by using the guarantee that a rename is atomic as a workaround.

It suggests a way to add metadata about messages through the use of flags added to file names, but doesn’t specify what these must be.

I don’t think, therefore, that Maildir as a standard provides enough to allow it to be relied upon for portability between clients (MUAs). That’s not to say a standard couldn’t emerge if there were some agreements.

Why not store all of the e-mails in a mailbox in an sqlite database and keep the folder hierarchy in a directory?

https://bugzilla.mozilla.org/show_bug.cgi?id=361807

IMO, I'd install a local IMAP server and only use that (currently dovecot).

Wouldn't it be better for it to store it in a Sqlite file so that it is easier for third party programs to query it?

Then you're creating a new format (sqlite is well known, but the schema won't be), which third parties can add support for; depending on your exact goal, this is probably worse than using an already-supported format.

Isn't Maildir too wasteful?

for personal mail the advantage in Maildir is incremental backups acting on the file system level are efficient. With an mbox file or sqlite db the whole file has to be backuped and transfered again. Of course for backup/snapshot mechanisms working on the block level one big file does not matter.

In what way?

Each email in its own file means a minimum of x KB on disk for each email, where x is usually 4. For people with thousands of emails that's very wasteful.

I would take a safe Maildir implementation over subtle (or sometimes catastrophic) data loss bugs any day. The one really serious problem I have with Thunderbird is that after being stung more than once, I feel like not only do I have to keep normal backups, I also have to retain old backups indefinitely, just in case a folder was corrupted silently a long time ago and I only find out months or years later when I need to search older messages.

Maildir, as opposed to emails stored in one big file, does cause some disk space to get wasted, but in past 10 years, it is immaterial. Even million empty emails, each taking 4kB of disk space, would take only 4GB in total. The small files aren't a concern for saving disk space: the big ones are.

Maildir has the advantage that searching, adding and deleting emails is very fast, much faster than if those operations were made on single big file. Worth it.

That's ~4MB per thousand emails; while I agree it's conceptually wasteful, I doubt it would have any real consequences for anyone.

Waste isn't the issue with big folders; filesystem performance (especially NTFS, which is to say Windows users, which is to say ~90% of users) degrades pretty horribly when folders have tens of thousands of files. Even on Linux filesystems, having a folder with a 100,000 files in it was having issues.

FWIW on ext4 I benchmarked adding millions of files to a folder and compared with `xx/...` and `x/x/x/x/...` and found the performance to be basically identical between the options and didn't degrade with the file count.

The one thing that I did find is that deleting folders with many files was very slow. I guess that path wasn't well optimized. However even then you could deleted hundreds of files a second so for regular mail usage it likely isn't a major issue.

Only if you use filesystem that cannot efficiently store small files, either together with metadata or packing several small files together, and allocates a full allocation unit for each separately.

Reply on news.ycombinator.com