Comment by crispinb | Hacker News Reader

Comment by crispinb | Hacker News Reader

Can anyone recommend tools for removing email duplicates? I have a mess of archived email from different sources I intend one day to consolidate onto a decent IMAP provider. Over the years I've done all kinds of forwarding from one account to another, been in & out of gmail,etc. It's a bit of a mess, and consolidation would result in me using much more storage than I need because of dupes.

A trustworthy deduping tool (whether operating via IMAP, or on local maildirs) would be useful. I've seen a couple around when I've searched in the past, but would like a recommendation from experience before running something over some gigs of archived mail.

I use mutt for that which modifies local maildir so it's super fast. Commands are: "D~=$". Then it's just a matter of syncing local changes back e.g. with mbsync.

Thanks. I suppose I should take another look at mutt. When I have in the past it's seemed a bit overwhelming with features & config. Does that command remove dupes in a single folder or a hierarchy?

It works in a single folder only. It's actually a set of the following commands:

  - D tells mutt to delete messages by pattern
  - ~= is pattern which selects all the dupes in current folder
  - $ finalizes the command

I think this [1] is where I learned about it and there's also a handy folder hook for mutt which automatically removes duplicates when you open the folder.

[1]: http://promberger.info/linux/2008/03/31/mutt-delete-duplicat...

Thanks. Could prove useful.

Reply on news.ycombinator.com