Comment by secure | Hacker News Reader

Comment by secure | Hacker News Reader

Parent Full thread

In my experience, it is not so much the installation itself, but the regular maintenance and the many subtle errors that can occur, which make running your own email server NOT as trivial as you make it sound :).

As for a few examples:

1. Mailservers around the world have different degrees of strictness in what they accept. Some require the sender’s mailserver to have a valid and matching PTR and A/AAAA (!) DNS records. Others don’t care. Some check black lists, some don’t. Some even resolve the MX record of your sender domain, connect to it and try to start delivery of an email to ensure your address is valid.

2. By default, postfix doesn’t warn you about undeliverable messages for quite some time (a week is the default, I think). So if there is any error in your config (e.g. I changed my DNS resolver config, then didn’t restart postfix), your mails will be stuck without any notice, for a number of days.

3. If there are SSL certificates invoked, they should be valid. Some mail servers will not use TLS at all, some will fall back to plain text, others will cancel delivery if your certificate is invalid.

These are just a few examples I have encountered recently, but every time they happen I am incredibly frustrated that emails either did not reach me (usually I detect that quickly) or are not delivered (detected only after a few days).

So this implies there is a market for a simple mailserver that just mimics gmail (or whatever) on your own server:

e.g.

apt-get install gmail-server

I've actually wondered this myself. Why isn't there a good preconfigured drop in solution for this? It would be especially nice if it had a good web front-end that you could log into remotely. How hard would it be to develop something like this? I assume it would be a matter of forking an existing project and adding some custom configs. The hard part would be the web component.

Freedom box has talked and hopefully worked on doing this, but Im not sure if they have gotten as far as improving the interface of a webmail yet.

Still, I do think there exist some very talented grafic designers out there that would enjoy playing around with a webmail interface. Hopefully, articles like this will increase the demand and interest in run-it-your-self webmail packages, so one would do that and not spend all focus on building more CMSes.

IIRC, most webmail packages now (SquirrelMail, RoundCube) are just web front-ends to IMAP. It stands to reason that something more integrated, is probably more difficult.

Search is IMO the hard part. I know you can do Lucerne, but getting that working well with email is hard. Until recently Outlook's search was crap compared to gmail. too, and mutt is still essentially "headers only".

Still, shouldn't be that hard to do as long as it is your own server and you trust it; I imagine a lot of the difficulty of gmail is scaling, and even a single user with 20-50GB of mail isn't in the same league.

If you were to outsource it to a service provider:

I'd love a way to do privacy-protecting search on my mailbox, either by building/maintaining a local index, or even more amazingly, some kind of cryptographic/data structure magic: do processing once either pre-encryption or on the local device, then add it to an index, with a configurable slider for data leakage vs. search quality. Search could execute locally and remotely.

(This is to allow you to use webmail with no local persistent storage, or a new phone, to search your email on a server, without trusting the server).

Gmail search is actually pretty spotty.

I download my email to the Mac mail app. That has let me find some emails that were unfindable on the web interface (even when I searched by sender email!)

What if there were images on, say, AWS that let you spin up a mailbox instance? Packaging up a standard "I want a mailserver" system might be doable.

doesnt solve alot the issues (blacklists, SSL etc.) and is quite costly for just running email.

But they are solveable. Nice little business idea here.

> and mutt is still essentially "headers only"

Mutt is more like "everything I can come up with from the command line." Sure, I won't make my grandma use it, but I've never needed any searching capability beyond grep and co.

Yeah, I end up grepping organized-by-years Maildirs, but that's somewhat suboptimal.

Ever looked at Mairix?

No, looks interesting, will check it out.

Not necessarily. Try to do something like write up your email in Markdown, and create a script to automatically convert it to multipart/alternative with text and html versions. So far as I can tell the only 'real' way is to insert the script between mutt and the sendmail command, but that's less than optimal.

notmuch for search.

http://notmuchmail.org/

Agreed. I've handled over 100,000 messages with it with no problem, and I know people that have handled well over that. Instant full-text search, no matter how much mail you throw at it.

How about a good old desktop mail client? I don't really understand why so many people insist on only using webmail and then complain about the lack of certain features and/or GMail lock-in.

There already are several solutions like this that have been around for years. Check out Zimbra. But still a pain to maintain and moving over to Google Apps several years ago, I never looked back.

Have to agree with this. I ran my own mail server for a few years and even though set up was easy, deliverability was a PITA (I was never able to successfully deliver to hotmail addresses even after doing everything possible on my end).

Then there's spam.. what a headache. Suddenly one day I realised that I have better things to do with my time and now everything goes through google servers.

Did you install greylisting and blacklisting? It really do reduce spam to ~0, and is a one time 5-20m configuration.

As for hotmail, I strongly remember doing a telnet to hotmail servers and succeeded in sending emails while writing commands by hand on my private computer. Thus without any special headers or other magic, I could send email without any issues. It would had been interesting to hear details on why hotmail rejected your emails. Default postfix installation?

I agree with both of you. Mail is hard, not easy. We should host our own email anyway, because the alternatives suck.

I wonder if there is a "Mail Admin Best Practices" doc floating around the web yet?

http://37signals.com/svn/posts/3096-behind-the-scenes-giving...

This is pretty good. :)

There have been a couple of "setting up a mail server" howtos come through here. I recall one that was huge in the depth that it went into (the depth also made sure that I never got around to reading it).

Subtle errors are a problem, as anyone who is a programmer or sysadmin will tell. It's a constant issue when programming in C or Javascript, and it's almost a job description for a sysadmin. Mail installations could clearly be improved here with testsuites that test the installation and not just the mail server code.

But to address some of those concerns.

While errors in the config can cause big problems, it's not very common for people to change it once it's installed. It's like doing changes to the kernel, Apache, or Xorg settings. Sure, things can easily be broken there but for the common case, the defaults are good once one has passed the installation wizard/guide.

In case of SSL issues between mail servers, I am rather sure that postfix only does a best-effort. Thus if your certification expires, they will fall back to plain text. However most CA's will warn you several times once it gets close to the one year mark, so it's not a very big workload to handle. If you know of an email server that will cancel delivery, please let me know as I would be very interested to hear it.

PTR is a real issue, but if you have static IP, PTR records should be rather easy to get. It is also a one time cost for the domain name. For a company network, a static IP and support is commonly included in the price. You email/call them, and they add a PTR. For private users, a VPN solution might be needed in some cases, but I'm unsure if it's that much of an requirement. Gmail seems fine in sending emails to locations without PTR, or receiving emails from servers without one. However I have not tested this fully or with other services like to Apple or Microsoft.

I only have a few small-ish mail servers under my care, and I couldn't agree more.

Reviewing the history of the mailop list [1] shows how frustrating it can be when a behemoth (AOL, MS, etc.) stops accepting your mail. It's bad enough when you're a sizeable mail operation; as a tiny standalone mailserver, you are not at all a priority, and it's possible you'll lose the ability to communicate with a significant fraction of the Internet's email users (regardless of how complete/correct your configuration) and with little recourse.

[1] http://chilli.nosignal.org/mailman/listinfo/mailop

Plus you need offsite backups, for both content and config... it's an escalating problem.

I haven't messed with the defaults on undeliverable messages, but my recollection is that Postfix sends you a "hey, it didn't go through, but I'm going to keep trying" email at 4 hours, and you get the final "I gave up" email about a week later.

Don't forget maintenance of the hardware itself, backups, and everything else that comes with running a server (assuming you didn't buy one in the cloud).

> assuming you didn't buy one in the cloud

Buying a mail server in the cloud is the most practical option. I doubt EC2 would peek inside your VM to censor mail as Apple is doing here.

Running a mail server at home can get complicated. Other than having to manage the hardware yourself, there are limitations on residential internet connections. Not only do residential ISP often provide dynamic IP addresses, they tend to block port 25 (at least outgoing) so as to counter spambots. Some spam filter may also treat mail delivered from your home server more harshly.

Though, if you really can't trust anyone (cloud providers, SMTP relays, or the network in general) you're be better of using PGP.

Reply on news.ycombinator.com