Hacker News Reader

Comment by lisper | Hacker News Reader

I use sieve as part of my home-brew spam filter setup, but I hate it. The syntax is atrocious and the documentation is abysmal. The only way I was able to get my setup to work was by copying examples I found on the web and incrementally tweaking them by trial and error until they did the right thing. It took days and the end result was only about a dozen lines of code.

How hard could it be to design a sane mail filtering language? I guess I'll just have to take that on one of these days.

View on HN

FDM seems to have a nice way to define filters:

https://en.wikipedia.org/wiki/Fdm_(software)

https://acampbell.uk/wp/2019/04/11/replacing-procmail-with-f...

https://hackriculture.fr/tag/fdm.html

https://manpages.debian.org/testing/fdm/fdm.conf.5.en.html

Seconding fdm. I have about 1,000 lines of rules to process when I fetch via POP3 and it has decent debugging facilities as well. Not to mention, it is fast and comes with a friendly – but powerful – syntax.

If I had the time I'd love to simply reimplement procmail. In principal its exceedingly simple as procmail doesn't parse structured headers or entity bodies; the expressions are applied to the raw lines. The fact procmail abstains from more complex parsing is probably why it's seen as few bugs as it has given the approach to parsing--classic Unix string slicing and dicing rather than a less bug-prone stream of scalar tokens which I personally prefer when doing string processing in C. (Can't overflow buffers when you're not taking pointers into and copying fragments every which way.)

Probably a good exercise as a Rust project, but I run OpenBSD so C (or C+Perl or C+Lua--Lua making it trivial to build a single binary) would probably be a better fit--no runtime or even build dependencies. An interim measure for OpenBSD might be to apply pledge and unveil sandboxing. I keep meaning to inspect how procmail loads and parses procmailrc to see if they could be applied early enough to provide realistic mitigations.

procmail is now on GitHub, so perhaps you could contribute there rather than reimplementing?

https://github.com/BuGlessRB/procmail

Aye, will check it out.

EDIT: The original author has resumed maintenance?

Correct, although I don't know how much maintenance it will get or for how long.

Iff you like Python, then mailprocessing¹ is a nice solution for this. You just write a small Python script to implement your rules:

    for mail in processor:
        if mail['X-Bugzilla-URL'].contains('zephyr-ds'):
            mail.move('Zephyr')
        elif ...

Given the layer it operates on you don't need to worry quite so much about processing errors dropping mails either.

¹ https://mailprocessing.github.io/mailprocessing/

How well does it handle non-ascii character sets or non-compliance emails? I haven't been able to find a mail processing library that gracefully supports something like Arabic characters in the headers/body and also does not include a content-encoding header.

Behind the scenes mailprocessing basically works by executing your Python configuration file with some injected globals to make processing easier, so you can apply any hacks you feel like to workaround breakages in the incoming mail.

In the specific case of Arabic characters in headers, you can see from the source¹ that when decoding fails an error is logged but you're still able to work on that header. How much work you're willing to put in to recover from that error by guessing charsets(perhaps via chardet²) is up to you.

The level of customisation that is possible is my favourite feature of this type of software. You can start with a config that is just a couple of lines, but it grows with you with no artificial ceiling imposed by the software author's imagination.

¹ https://github.com/mailprocessing/mailprocessing/blob/master...

² https://github.com/chardet/chardet

Not a huge fan of sieve either. For spam filtering, I've spent some time over the last few years tweaking spamassassin weights and have had pretty decent results with that. I think this was the one I started with for SA which was pretty decent then did some additional tweaks based on what I was seeing come in https://gist.github.com/ishmaelen/6c7a209dd00f9d2b968fd2bb0e...

Roundcube has a sieve plugin and makes it very easy to manage Sieve rules.

I don't need webmail at all but it's useful just for this.

Reply on news.ycombinator.com