Hacker News Reader

Comment by api | Hacker News Reader

1. Running a mail server is an unbelievable pain in the rear from an IT perspective. Note that by "mail server" I mean a good setup with spam filtration, webmail, SMTPS, IMAPS, etc.

2. Really good rich webmail. I personally use Mac Mail.app most of the time, but having that rich webmail is nice.

3. Filters mail at the server side.

4. Very good spam filtering... I post my gmail addresses on web pages with no obfuscation and get maybe 1-2 spams per month.

Cumulative: it's one more thing I don't have to jerk around with. It just works.

View on HN

I disagree on most points...

1. It takes a few hours to set up, sure, but it isn't difficult – setting up a mail server on a *nix box isn't exactly uncharted territory, every possible aspect of it is documented to death on the web. And once it's up and running it takes practically no effort to maintain. Yes, even with spam filtering that keeps my inbox spam free, despite my address having been used in several public newsgroups and forums.

2. It's a matter of preference. I too use Mail.app primarily, but I like being able to SSH to my server and read my mail in Mutt. As nice as Gmail's UI is in many regards, it's useless for threaded discussions on mailing lists.

3. Err, ever heard of procmail or maildrop, among (many) others? This is one of the major advantages of running your own mail server, actually; Gmail gives you very limited filtering options, especially when it comes to mailing lists.

4. Gmail has great spam filtering, but I find my Postgrey + SpamAssassin setup to work just as well.

And in my experience, OP has it right.

1. Setting up the mail server is not the issue. It's the ongoing maintenance to ensure close to 100% uptime so you don't lose any mail. Having worked as a sysadmin maintaining linux boxes for several companies, mail server maintenance accounted for the vast majority of the hours we spent working on servers. This doesn't even account for the hardware upkeep to keep RAID sets in good shape, tape backups, etc.

2. There is simply no open source project that offers a UI anywhere close to as good as gmail. Yes, someone could take on that project, but it will be years before they achieve a similarly polished result.

3. Sure you could roll your own filtering that would be more effective than gmail's, for you. Again, this is further work that you have to do whereas gmail "just works".

4. SpamAssassin is one of the great open source projects in my opinion. Combined with ClamAV you can get a really robust solution that works fairly well. However (afaik) it lacks the learning heuristics that gmail's huge body of data can provide to keep even the newest template spam out of your inbox.

This is not to say that I don't agree with some of your points, but in my opinion the OP has a valid perspective, which I suspect many IT people who value their own time will share.

I do value my own time, and I'll say it again – post setup, I really spend no appreciable time keeping my mail server running. My VPS provider handles all hardware issues (with zero downtime in the last two years), backups consist of a nightly cron job and require no manual intervention, and the last time I touched my Postfix configuration was over a year ago.

"Rolling my own filtering" consists of placing a single mailbox_command line in the Postfix configuration file, it hardly qualifies as a time sink. Writing mail filters for procmail takes no more time than doing so in Gmail, the difference is that the former lets you write much more expressive and fine-grained rules.

And yes, SpamAssassin does have a built-in learning Bayesian classifier. Which I previously used to learn from anything that I manually classified as spam, but these days I don't even get any spam to invoke it on. See sa-learn(1) for details.

You are missing a very critical point: either you need to rent a virtual or real server and spend time configuring and maintaining it - or you could try and dare to run your mail server from at home on some linux box which will open a whole can of worms...

So gmail gives you an easy and free alternative and you can just sign up and access mail via web, pop3 and especially IMAP, which made it superior to pretty much every single other webmail when they launched that feature - and I believe that is still true today, most free webmail providers don't give you free IMAP. You are "paying" by giving away your private information but evidently, people prefer it to setting up their own mail servers.

I'm not sure where you're spending time on maintenance. I set up my current mail server 3.5 years ago (postfix, spamassassin, dovecot, sieve, roundcube) and barely ever touch it -- the only maintenance required is the very rare security update.

I use tarsnap for backups, so no tapes required.

I don't love running my own mail server, but as a hacker, I don't see that any of the alternatives are better.

Regarding point #1, I have maintained Linux mail servers both for large shops as a professional sysadmin as well as for myself and small organizations, and each has been a very different experience.

In the large environment, mail server maintenance does take a disproportionate amount of time. But that's because the environment is so complicated: there are several layers of servers, mail is stored on NFS so there are weird locking issues, LDAP is used for user accounts, etc. There are some very bad failure semantics and if we're not careful, we start bouncing mail.

My small setups use exactly the same software (Debian, postfix, dovecot, spamassassin) but the environments are much simpler: one server, no NFS, no LDAP. I use a VPS so I don't have to worry about hardware. Backups are an off-site rsync in cron. Failures rarely happen but when they do the semantics are much nicer: temporary failures so the mail is resent later. My VPS host failed for the first time in 5 years recently, so I couldn't get mail for a few hours, but I didn't lose any mail and didn't have to spend any time on recovery.

I also run other stuff on my VPS, so the monthly cost and recurring maintenance overhead (security updates/backups) is amortized. Adding mail as another service doesn't change these costs.

I'd love to ssh to my server and use some modern MUA like Sup, but there are lot of places where you cannot ssh through local firewalls.

When you work parttime as a consultant or when you just meet with some potential customers/investors/cofounders/whomever you often need to use THEIR infrastructure. So what then? Do you imagine that you'd be unable to get an email from the people you're talking on the spot just because you need to tunnel somehow to your server? Bit weird..

One option is to always use own mobile connection, but still there are places where there's no carrier (like some server rooms), or you have to use customer's intranet at the same time you're relying on what's in your emails..

A handy trick I have always used is to get a second IP address on my servers and set up ssh on port 443. From memory I have only had one case where a firewall blocked it.

The one that got me was spam filtering. Google does a pretty good job at that, and I was really drowning in it.

I think they deserve a lot of credit for creating a product that both my mother (definitely not a 'power user') and I (previous email client: Emacs' Gnus) can both be reasonably productive with.

It's really easy to install SpamAssassin and point your MTA at it. Even the default configuration is highly effective.

I did that, back in the day, but it just wasn't enough.

One could always mooch off of Gmail's spam filtering... Use SpamAssassin in conjunction with a gmail account that is only used as an attempt to attract as much spam as possible. Then just download the spam folder over IMAP and use it to teach SpamAssassin...

Gmail's spam filters are so good because they have a network effect going though, I doubt one spam folder would achieve much, you need a much bigger sample of all the spam than one account could ever attract

Pretty sure you are making some unwarranted assumptions here – other commenters mention that SpamAssassin can download updates to itself regularly, so you aren’t starting from scratch with a completely untrained Bayesian algorithm.

Respectfully, no.

The primary issue in spam filtering is not that we don't have training corpora, it's that spammers are very efficient at finding holes in your system.

One example is when the Hotmail team first enabled keyword filtering. When the spammers found out experimentally, they began injecting HTML comment tags into high-weighted words like "free", breaking the model. When the Hotmail team took steps to combat this problem, the amount of spam that employed this technique dropped from 5% to close to 0%[1] in a matter of days.

Spam detection is complicated and hard.

[1] Hulten, G., Penta, A., Seshadrinathan, G., and Mishra, M. Trends in spam products and methods.

I'm not saying it's perfect. Obviously having a network effect allows you to do things like flag messages that are exact duplicates over multiple sent-from and sent-to addresses.

I've seen this happen where 1 or 2 messages hit my inbox, but the next 20 or so are in my spam folder.

"The primary issue in spam filtering is not that we don't have training corpora, it's that spammers are very efficient at finding holes in your system."

Finding holes on sites like gmail and hotmail is actually much easier than on private sites, because the spammers no doubt have accounts on gmail and hotmail and can test what gets through and what fails and tweak their algorithms until their spam gets through.

Spammers don't have that luxury on private servers, so they have to spam blindly. So in this respect private servers have an advantage over gmail and hotmail.

On the other hand (as has been pointed out many times in this thread), gmail and hotmail have the advantage of advanced spam detection algorithms and virtually instantaneous feedback from millions of users.

> So in this respect private servers have an advantage over gmail and hotmail.

Besides, spammers care more about being able to circumvent Gmail's spam filtering than they care about circumventing your own system.

That's a perfect example of how spam detection is harder for a centralized server operator like Hotmail or Gmail than for somebody who's running their own server: if I were to add a delete-HTML-comment-tag preprocessing phase to my own copy of SpamAssassin, spammers would have a difficult time detecting that, and so would not be able to adapt. (Because none of the dozen or so people who can get mail on the server are spammers.)

SpamAssassin takes care of this for you by automatically downloading new filtering rules. You host the spam filtering software, but the actual rules are out-sourced by default. It even includes configuration for multiple DNSBLs, DNSWLs and RHSBLs by default. You can add your own local rules if absolutely necessary too.

I use SpamAssassin server side, and Thunderbird's own filter polishes off the rest. I see perhaps 1 spam mail per week (from the literally tens of thousands that hit my server).

It's still a service which you have to maintain and there's constant config & upgrading required since some spammers are smart enough to test their messages with it first.

I've run SA professionally for modest (low hundreds) of users. It works reasonably well but it's a job and I'm not paid to do it personally.

SpamAssasin will auto-update its rules if properly configured. I have touched my SA install in years.

nupark2 is correct. There's less maintenance involved in running SpamAssassin than there is in running Firefox.

Would you pay for this? For the last 3 years I've been creating a platform for this kind of servers for a small spanish ISP. It has grown to 15 Openvz hosts. A rails app connects to the host to create the virtual machine. Centos is installed and configured with puppet. A rails app is installed in every server and used to configure the accounts, domains, etc.

Postfix, Cyrus, IMAPS, SMTPS, POP3S, HTTPS, Sieve for filters on the server, roundcube as webmail (on a new product I would pay for @mail), decent spam filter (right now we run our own system based on commtouch and spamassasin), etc. You can export the whole cyrus mail storage folder or export mail to other server with imapsync, same thing with the database that contains the account, domains and aliases, so you are not locked. Billing can be done by hard drive space or accounts. They are also good as outgoing mail servers because you are on your own ip.

I run everything by myself, right now we have 126 servers with about 12300 accounts and thanks to puppet it's really easy to admin. I've always thought that I should try to do it by myself.

Why pay when you can get it from Gmail? :)

Aren't you a little bit uncomfortable giving Google all that data? Do you keep a back-up??

Mail.app is keeping a local backup (using imap)

Not even a little bit.

Total number of times Google has lost my data: 0

Total number of times I have accidentally lost my own data: Way more than 0

Total number of times Google has in any way caused me harm by having "access" to my e-mails: 0

Total number of times I've been glad Google used intelligence gathered from my e-mails: Many (pre-populating Google+ circles, training a great spam filter, Priority Inbox, great search, decently appropriate and well-targeted ads)

What happens when Google suspends your account and you loose all your email, docs, etc? Your online identity (you no longer have your main email address) gone?

Own your MX record and use forwarding and/or Google Apps for Your Domain?

That's what I do, and it gives you a portable email address, but doesn't address the other concerns. Ultimately, email is a communication medium, not a filesystem. Using IMAP folders for long-term storage is tempting, but it's off-label.

You only have ~100 email accounts per server?

yes, it's pretty stupid but it's the product that the commercial team sells (your own small managed private mail server). Some of the servers have more than 2000 accounts but others have 6 or 10... A few years ago, until 2006, the servers where small appliances installed on the client's office. Now they work the same way but mostly virtual and since most of the clients are small bussiness.. some have really few accounts.

On our shared mail system we have about 40k accounts in 2 cyrus servers (with 2 more for HA)

The primary reason I won't outsource this service is trust. I don't trust anyone with that data.

Access to my email allows for resetting all my passwords, seeing all my contacts, and tracking literally everything about my day-to-day business and personal life.

I hope all your email is encrypted, because otherwise every hop the email traveling to/from your server takes has access to all the data you hold so dear.

Actually most SMTP now talk SSL to each others, so while the hops (generally, just 2: sender and receiver) could also be compromised, the attack surface is smaller than one would think.

"most SMTP now talk SSL to each others"

Not to dispute your claim, but how do you know this? Is there some sort of global, internet-wide survey of SMTP servers somewhere that supports what you say?

It's easy to tell which mail servers are using SSL by looking at your mail server logs. He probably only cares that most of the mail servers talking to him use SSL (which is also true for every mail server I have administered).

This is a much smaller attack surface than a centralized data store holding static e-mail archives and accounts for a large number of users, open to at-will offline browsing.

DNSSEC and SSL further reduce this attack surface.

I haven't used Mail.app since an incident about 2 years ago where it deleted a bunch of my messages. I was moving them from one folder to another when the connection to the server dropped.

I don't know why this should happen with IMAP (it is possible to move messages from one folder to another atomically) but it did.

Reply on news.ycombinator.com