What's your setup for your email account these days?
I have my own physical servers, that I built and tested myself, that I'm colocating. They handle both incoming and outgoing, and I've been doing it for so long that there is no previous reputation for the IP addresses I use.
In spite of how vehemently some people, Reddit's /r/sysadmin, as an example, want you to NOT host your own and use issues like deliverability as reasons, it's really not hard at all. It's super simple to refute all the major points they make, because they're so painfully weak that anyone that believes them may actually not have the aptitude to do it, and therefore shouldn't be telling others to not do it.
1) The primary issue brought up is deliverability. If you don't have a static IP, or you don't have control over your reverse DNS PTR, or the reputation of your IP is poor, then pay a company to smarthost your outgoing mail through them. It's a few $ a month, and poof! Problem solved.
2) There is no problem 2! Incoming email is incredibly straightforward. Even if you're on a residential network that blocks incoming port 25, you can pay for a VPS or something like that on a public address and port forward to your mail server.
Why make a distinction between simply hosting your email with a VPS and doing this? Well, one of the primary reasons for people hosting their own email is being able to possess your own email - that is, your email isn't sitting unencrypted on a server that you don't control.
I've even run an email server in my car while driving across the country, just to show how easy it is. It uses tinc to forward a public address and had no issues with email in either direction :)
> your email isn't sitting unencrypted on a server that you don't control
This is valid if all the other parties that you send emails to self host their email servers as well. If you have your own self hosted mail server but you talk to gmail people then "your" emails are going to sit unencrypted on Google servers.
Still using sendmail ;)
> This is valid if all the other parties that you send emails to self host their email servers as well.
I used the word "sitting" deliberately. Of course we can't control the rest of the world's servers, and we can't even fully control those owned by others but rented to us, but we can control our own.
I'd add that it's not just the reputation of your IP, but the reputation of your whole /24 block. That seems to be where a lot of filtering based on reputation takes place. I've got my corporate email server on a VPS (running FreeBSD with exim) with a hosting company that cares about its reputation (and charges a bit more for it) and email delivery is never a problem. I've also got a identically set up but personal email server on a crappy little cheap VPS which does have a static IP and hasn't had any spam on it for years, but plenty of other IPs in the /24 block get filthy with spam. My IP gets blacklisted quite frequently because of it.
That is due to the prevalence of what is known as Snowshoe Spam.
Spammers get old of (or infiltrate) large net blocks and then spread out their spamming across the entire range.
The extreme version of Snowshoe Spam is known as Hailstorm Spam. For fairly obvious reasons ... spammers get hold of (or infiltrate) a large netblock, intensively abuse the hell out of it and then move on.
Be sure to search for "smarthost" in quotes, because "smarthosting" is a marketing term.
I have a spreadsheet where I compare what it would cost me to do this in a cloud or on VPSs, and though there are some squishy factors (I enjoy messing with servers occasionally, when I have to visit the colo I also visit old friends in that area, etc.) it's still cheaper for me to keep my physical servers in my colo.
> 1) The primary issue brought up is deliverability. If you don't have a static IP, or you don't have control over your reverse DNS PTR, or the reputation of your IP is poor, then pay a company to smarthost your outgoing mail through them. It's a few $ a month, and poof! Problem solved.
By that "logic" self-hosting is easy because I could have a third party handle outbound AND inbound mail, which is essentially standard mail hosting with a custom domain.
Plus you cannot just turn-key create a high reputation, you start out with low and your emails will be dropped into spam.
Poof is right. There goes the primary reason people want to host on their own.
I know of two primary reasons, and neither disappears because of smarthosting.
I would argue that the overhead of setting things up is more pricey then paying for smarthost forwarding, but maybe I'm too practical,
I don't know why there isn't some kind of pre-made install that you just click the "OK" and it downloads and installs the email servers.
When I looked at it last, which was about 3 years ago, you had to install 5 or 6 or more different apps, get the interoperatbility going, setting different switches....it was endless. All I want to do is click an "OK" and it is done.
Quite frankly, I don't understand how people can write very technical apps like email servers, yet be so completely unable to create a one-click installation feature. This goes with so much more than email, by the way. It seem's like it would be so exceedinly simple to do one-click setups.
But I am curious, how exactly did you learn? Did you work with someone who already knew how to do it so you didn't have to learn on your own? Or did you learn 100% on your own?
I'd be very curious to learn from you on the above issues.
Right now, they have to be technical (or at least pay $$ to tech people) to get that going, and it does slim down the numbers a little.
Getting the software lined up is the easy part, truth be told. It's the anti-spam/malware/anti-relay etc etc that is the challenge. And most big Email companies weigh how long you've been around, so you also have to be patient for your reputation to get high enough not to be in people's junk boxes all the time.
I continue to do so for several reasons, hosting multiple domains, ad hoc addresses, etc, but the biggest surprise reason is a completely not email one. It hosts DNS.
I LOVE hosting my own DNS. The first and most important reason is that, unlike all of the other services, there is no delay getting new records on to the net. Until MIAB, the 'will be updated in 20 minutes' thing drove me nuts. It made doing programmer stuff that relied on DNS incredibly annoying. Now, my LetsEncrypt proof is there instantly.
Of course, not being reliant on the good will of Google or having to decide if it's worth another $3.50/month for another address are all good things. But, the other main reason I choose to host my own server is, also why the MIAB guy makes it possible, to cast off the shackles of the corporate overlords.
It is bad to allow a mission critical function to be owned by five big companies. Much like the current consequences of having relied on Twitter for our short form public communication, I see the move to the decentralized Mastodon as similar to my implementation of MIAB. It is, for me, a blow in favor of internet freedom and robustness.
Running BIND is very simple, and there're lots of how-tos and some excellent O'Reilly books, too.
You can run your own DNS on a VPS, on a machine on a static IP, or even on a residential address, if it doesn't change too often. If it does change, you can run your primary on your home and have a public machine be a secondary.
It opens up so many possibilities :D
My suggestion is to get a new domain and then try it with that. Once everything works, and your IP reputation is in the clear, you can port your other email domains as well.
[1] Historical issues with running at an ISP in the late 90s and the various security issues since.
I agree with the other commenter that running BIND is not too difficult but, if you are going to run a server, MIAB is a good way to kill both birds, mail and DNS.
Also, the MIAB DNS UI is good.
It does seem that it can depend somewhat on what IP you roll. I guess if you get a good one and stick to it you may be able to scrape by with just the ASN reputation holding you back.
Source: I run a service that sends email on Digital Ocean and I have to use third-party services to send to many hosts. (in fact I tend to whitelist inbox providers slowly once we have built some domain reputation)
I haven't had a deliverability problem in years. Maybe it's because I got better at the SPF's, DMARC's, etc.
I will say that I think it's an internet sin that these big companies make it hard for individuals to run their own servers. It seems like an attempt to make a corporate monopoly. My server has never been used for spam a single time. There is no reason I should not have been able to remedy problems instantly.
Also re Gmail etc that is also true, but the difference there is it's limited to a couple whereas do also emits general abuse under the guise of "researchers" without identity or contact details etc - until they stop doing that they'll forever remain dropped entirely
Mailinabox just works, and I use it for my whole family.
I spent hours trying to diagnose this, and so did the hosting service - to no avail, because we'd try something, a delivery worked ... and the next day, to a different recipient, it didn't - and it is sometimes hard to get feedback about those things.
I really wanted to support the smallish hosting service - but I couldn't afford to lose so many deliveries. I sadly switched to FastMail and not a single mail that I know of was lost for a few years now.
You say you are investing 2-4/year troubleshooting delivery issues; How do you know that the things you don't troubleshoot actually reach the receipient's inbox?
I love this feature so much. It also integrates into 1Password and you can automatically create a masked email on any signup. I exclusively use it now. It's awesome
Their spam system works great but needs some breaking in: to train it, you need to have marked and deleted 200 spam emails. Only when you delete spam, the system updates its filter. I had some annoying spam that kept landing in my inbox, I had to create an automated "send to spam" rule until the personal spam filter started kicking in.
The only issues I've really had are general custom domain & address choice issues:
Choosing to use me@myfullname.tld was not the best idea. A very tiny number of sites stupidly reject the use of a 2 letter username in the email address. Easily solved by setting up an alias of mail@myfullname.tld though. If I were starting over I would have just made mail@myfullname.tld the main address (and I would have aliased male@myfullname.tld). That also makes it a bit easier for humans to read too. Only 2 letters before the @ blends in too much.
Address hiding that many sites do fails to hide anything. Many times when you log into a site, it will display your email address in a way that's meant to avoid people seeing it over your shoulder. But most sites that do that only hide the username. So I'll see things like m*e@myfullname.tld
So I enjoy the fact that my self-hosted solution only rejects emails from blocked senders or DMARC rejects. At the very least everything else goes to spam so that I can see it.
I'm not opposed to hosting my own email server, I'm just too lazy to do it.
Mail deliverability has, with some minor exceptions, never really been an issue. These days you need SPF, DMARC, and of course a clean outbound IP reputation, but that's all rather manageable -- biggest deal is setting up strict filtering and rate limits on outbound messages, plus ensuring mailbox passwords can't be brute-forced (complexity rules plus blocking abusive IPs).
The only real recurring issue is people setting up forwarding to gmail.com/outlook.com/whatever, and the target service then becoming temporarily upset due to all the 'spammy' messages being sent: DMARC helps with that up to a point, but not perfectly, and directing users to reverse the mail replication direction (setting up gmail.com to pull, not my server to push) is a common chore.
For the current setup, the biggest headache was finding software with an acceptable webmail and domain setup self-service web interface. Next up was shared access to large mailboxes, Outlook calendar support (which just really can't be done with only CalDAV, even though that should be possible), and (believe it or not) getting 'Drafts' folders to sync across devices.
Since these companies are too big to communicate with us lowly humans, I've given up on the idea of them adding an option to allow this by designating a forwarding domain so the forwarding email server doesn't get punished for forwarded spam.
The fix is super simple: I've set up local accounts for people, and tell them to configure Gmail / whatever to use IMAP to fetch email from the local accounts :)
If they are supposed to be used by humans then prefer length to complexity.
The headache is with people who want to do things which aren't in my "acceptable use" envelope. I know there are adversaries out there ranging from the simply venal and self-righteous to the outright evil. That ranges from spam to tracking to phishing to malware delivery; and from scanning for open relays to credential stuffing to SYN attacks and more.
I actively map and target adversary infrastructure (and sometimes the "friendlies" are useful idiots). I use and encourage the use of 1:1 email aliases (I wrote and support TruAlias). I mess with DNS; I mess with L2. So around here if something doesn't work, you'd better ask me or your designated contact if you're expecting it to. There is no privacy around DNS or netflow info on my network.
My support issues are largely people-driven / political, which shouldn't be surprising given the above. However overall the support and IR load is light and the most problematic and chronic interlopers are the self-entitled aaS providers themselves.
I believe we need to keep doing this sort of thing, more.
I haven't had the same problem as people widely complain when hosting their own email. FWIW since you ask, I put this down to:
Using a quality ISP. VPS is ok, but not from widespread bulk providers. It seems to me that receivers judge quality by the network (eg. AS number) it originates from, not IP address. So you're judged by the quality of your neighbours and what those IPs are doing.
Properly set up SPF and DKIM. Someone here was recently stating how impossible it was to host your own mail, but a commentator quickly showed they had misconfigured.
Switching off IPv6. I love it in principle, but in practice there are big providers breaking IPv6. eg. Hotmail broke theirs a month or two ago. Or applying more stringent constants to IPv6 receipt.
I don't relate to people who say you need to "warm up" an IP address. It seems they might be often trying to use bulk VPS/cloud providers who probably get a lot of abuse. I don't see it's in the receiving ISPs interest to constrain IP addresses which only generate small amounts of mail (by definition this is unlikely to be spam), and it would be easy to mistake the per-network reputation for this effect. Make sure you're not on the various public spam/block lists, though.
So then your only concern is backup and email message export. For this, I setup automation with a script called got-your-back (gyb)[4]; it's a nice Python script with incremental backup that can archive your Gmail account and restore it to another account. I set up a second Gmail account to test this restore functionality.
[1]: https://blog.cloudflare.com/introducing-email-routing/
[2]: https://www.cloudflare.com/products/email-routing/
[3]: https://jhart99.com/cloudflare-outbound-email/#outbound-emai...
At the moment, I use the really good Migadu Micro plan.[1] It has very nice limits for the $19/y price (half if you're a student like myself), with a pretty lax and understanding policy. They also care about standards and make an effort for you to configure your domain with all the right records. Even their support was pretty fast and helpful, even in this plan where it's best effort. I am completely satisfied by their service and recommend it to anyone wanting more control over their email, but don't want or can't run their own server.
I've heard great things about FastMail as well. In the end, I think services like these are a great compromise between independence and convenience.
Never worry about deliverability. If your DNS provider has oddities that make it annoying to setup they will run your DNS for you.
For $30 a year (domain plus Migadu) you control your email domain. No possibility of being randomly banned by Google either.
I started on grokthis.net, they got acquired by Rackspace, stuck w/ rackspace longer than I should, groomed an elastic AWS IP for a year, got reverse DNS Mapping on it, and it's been pretty good.
Years ago, I was blocked once a year, and have not really seen much deliverability issues. More recently, either due to the reputation, or AWS being more diligent in blocking abusers, or blocklists being more focused, I have not had many if any issues. DKIM, SPF, DMARC all setup help.
Rspamd, postfix, dovecot, and roundcube are the tools I use to manage it, and it works for my pretty light load. There was a fun incident early on where my Bank did not send a 'Date: ' header (which is legal, per spec), but an rspamd default rule scored that as a high spam signal.
It's helped me learn about SMTP and all the related tech, and for someone who's in systems + operations, it's not that heavy a lift to do on the off time.
Several domains, several aliases, but I'm intentionally the only user. Because if I would hand out aliases to people they'll inevitably start receiving some spam there, that then gets forwarded to the {hotmail, gmail, ...} that they're alased to, and when at those termini it can be subsequently marked as spam. I surmise that that could be bad for my mail server's reputation and that's why I don't hand out aliases. That's just caution. There's plenty signal for the top-10 mail hosters to tune their reputation heuristics to so that that wouldn't happen, but I don't trust that they worry enough to invest smarts in improving the general state of email delivery. The incentives are not there. They're the incumbents, and interoperable federated services were good news when they were starting up (whooo hey we bought Postini and now we're doing this web based office thing where you can send and receive email, come join us in the beta, free forever, promised!" aka GSuite), but now that they're incumbents it's better to stamp out those cooperative federated protocols. Before anyone gets any ideas that you could actually own your own data and and could communicate with people over the internet independently of some FAANG-size corp, you know.
I wouldn't go back to use something else, what I much prefer: - I'm in control of my email address which is at the center of most of digital life. None of the big provider are up to the task, they do not care at all about me, they do not care at all about the impact of their mistakes (if they decide to block me). - I can manage my own backups much more easily - I can monitor and troubleshoot (with a 3rd party you are totally blind when something doesn't work) - I trust myself not applying any censorship (hotmail and gmail will take the decision to make email based on spam assumption disappear without any visibility from the user)
The downside: - Depending on where you host your system, you can have some issue due to ip reputation. It is something to be careful with before setting up a server - It requires a fair amount of testing, with different providers to ensure that your emails are delivered. - It takes a bit of time. Not too much, but it is still a commitment as, when it goes down, it's quite problematic.
* Mail in a Box for a bunch of low-volume domains, hosted on a major cloud provider VM. Total SMTP volume ~2k outbound emails a month. This was previously hosted on my own server in a datacenter.
* Plesk/Postfix for a higher volume domain tied to a nonprofit, SMTP relayed out through Amazon AWS. Total SMTP volume ~100k emails a month. Likely moving this to a standalone Mail in a Box installation as well.
Neither has given me any deliverability issues, but I did take the time to register with Google Postmaster, Microsoft's similar service, etc. The only problem I ever encountered was a spammer getting hold of a transactional account last year for about a week -- Google put a hard bounce on not just the server IP but also all domains tied to it. Took about 2-3 weeks before they decided to trust it again.
Would someone remember the name? I've tried to search for microsoft's postmaster but did not find anything.
For the past five years I've been using mail-in-a-box. (http://mailinabox.email) It is generally problem-free, but problems can arise if you host it on a sub-par provider (or provider network).
I highly recommend it!
It can be death by a thousand cuts though but if you can get it into a stable mode where no-one is blocking you, it'll generally just tick along.
Setup is exim on two machines with rspamd (aggressive settings) feeding one IMAP store which is accessed via an OpenResty proxy (lets me write my own auth stuff) on the other machine (and also controls where each user's IMAP store comes from - just in case I need to split and/or restore to another machine.) Oh and currently snappymail for a webmail frontend.
Main problem I had was fail2ban blocking my users because they forget passwords, don't have PTRs, etc. (it's set to be very aggressive on exim and dovecot failures because there's just a constant stream of probes and attacks) but I solved* that by unbanning the IPs people have successfully authenticated from in the last day, IYSWIM.
[edit: forgot the webmail and spam parts]
One day I ran in some problem with backscatter causing my mail server to be listed. It's good to setup your server to be less prone to this, I wrote a blog post with some configs / tips:
https://willem.com/blog/2019-09-10_fighting-backscatter-spam...
In general I suggest NOT to chose providers offering "safe" services, the more safe/privacy protected etc they claim to be the more likely they do not. What it count is:
- having your own domain, so you can switch from one provider to another NOT change you mail address, all your contacts will not notice the change;
- having mails accessible with standard protocols so it's easy to grab them, for instance GMail IMAP is crap, Tutanota do not offer standard protocols, you are essentially bound to them, Proton offer JMAP witch is NOT proprietary but still not much spread to have a good support and so on;
- having you mails on your iron, you can sync them with OfflineIMAP and nothing else for mere "live copy" on a home server and eventually back to another IMAP if needed;
- if you can USE them with a personal MUA, no matter if a WebUI or TUI or whatever but something local so in case of trouble on the upstream you are still partially operational and if you change the upstream provider you keep your usual UI.
All the above are from very easy and cheap to moderately easy and cheap (at least in absolute terms). Other options might be far less easy and less cheap.
Dovecot and Postfix, SpamAssassin and ClamAV; greylisting, SPF and DMARC but not DKIM. I handle mail for my family, some friends, and a few mailing lists.
Look at the bit about "backup MX for a remote site".
The specific thing you are asking for, however, isn't a backup MX so much as a second delivery to a local mailspool, which is also doable now that you know what it's called.
It’s pretty easy over a TCP connection (SSH for some reason didn’t work for me, so I’ve set up a Wireguard VPN between my hosts - also allows me to replicate KeyDB for Rspamd without having to set up a PKI for it), and works like a charm - backup MX receives mail if primary is down, and uses local Dovecot replica for delivery. When primary comes back online dsync makes sure they’re both consistent copies.
If anyone's got any tips or contacts for getting off of Microsoft's (live.com, outlook.com, hotmail.com ,etc) blacklist, I'd be much obliged. Their webform for such matters goes nowhere, and postmaster@ goes unanswered. It apparently has to do with the ASN of my hosting provider, but I've been on the same IP for 6+ years now and have done nothing untoward with it.
In the grand scheme of things, I really appreciate this setup. All my domains can contain an infinite number of derived addresses, which makes it possible to control outgoing information. For example, it's easy to detect and eliminate multiple types of spam sources.
I use Gmail, Hotmail, Yahoo for stuff I don't really feel is personal but potentially important, useful or otherwise interesting.
Recently I've been using docker-mailserver with my web apps and that's great too.
Xeams is monolithic (has all SMTP/pop3/imapd/... integrated), using it for 1 domain and a bunch of email accounts.
I access it from K-9 on Android or from a Web-Gui (Roundcube).
Then you can even continue to use the new mail providers webmail, you only need to use an IMAP client once! :)
[0] https://gitlab.com/simple-nixos-mailserver/nixos-mailserver
If you do have multiple users, then you also need to protect them against e-mail spoofing. This can be done by enabling SPF checks for incoming e-mail.
For live servers with multiple users, you want to limit the number of e-mails being sent per minute/hour to something reasonable, because some unfortunate user is going to get their password hacked sooner or later, and then a hacker might abuse your server to send e-mail unrestricted, which can in turn get everyone else blacklisted.
E-mail security measures has a lot of AoE damage, because when someone gets hacked and spam is sent, receiving servers don't just ban the offending sender e-mail address, instead they typically ban the IP of the server, which will DoS all users of the server. Extremely inappropriate, and it should be illegal, nevertheless that is what they do, and to a certain extend perhaps understandable. But, this is why you place limit on how many e-mails users can send, as it hopefully avoids that issue.
Some hosting providers has insecure server images. E.g. Ubuntu where the root user has no password, since, as they might argue: "you login with key file on SSH anyway"; however, custemers might not realize or remember that anyone can login on the e-mail server with the root user if the port is opened, and the server will become an open relay, as automated tools find it with port scanning.
You also need to make sure that whoever is logged in, and sending e-mail from a given e-mail address is actually the owner of the address. Postfix does not do that by default, meaning that users can just claim to be whatever@yourdomain.com, and there will be no check. Thankfully, you can not claim to be b.gates@microsoft.com, because that will be rejected due to DNS records. In the past you could because there was no check for that..
A lot of issues is of course prevented by simply limiting IP access to your own personal IP, and that's a good idea if you are the only user, but not practical when you got other users.
Ubuntu is not insecure regarding root user access.
Reference
https://discourse.ubuntu.com/t/security-users/11881
"This does not mean that the root account has been deleted or that it may not be accessed. It merely has been given a password hash which matches no possible value, therefore may not log in directly by itself."
First thing I do when moving to a new server is check whether the new box's IP is black listed in any mail-blacklist-dns service. If not, it's good.
What I have also found is that Gmail and other services are relatively forgiving on your server's IP, but marks mail as spam based on context and content.
In my new company we are using Gmail though. Easier to manage for non-tech people and it’s fully managed.
It's been a reliable (for the past 4 years I've used it) solution, with easy setup and laughably small cost. Currently it costs me about $0.3 per month to host personal mail for multiple domains (only one having some moderate activity in fairness).
All of these are mostly personal emails, I haven't tried to send large amount of emails to test their outgoing policies.
The main headache? Anti-spam. A lot of IP ranges are just outright broadly blacklisted (e.g. residential IPs, some hosting services), but also a lot of individual IPs reused by hosting providers got blacklisted for actually sending spam. Then you have trust scores which are a huge chicken/egg headache (how do you gain trust while being span-binned from the get-go?). You'll spend your days asking to get unblocked from various random third party providers that you may never have heard of (e.g. email a small business about their online shop, and their email vendor bounced you because they never heard of you, they take ten days to respond, and now it takes two weeks to email some random small business).
Plus the ever-changing requirements (and they ARE requirements, if you lack reverse DNS/DMARC/SPF/DKIM/TLSA/valid certificates you will be blocked).
It is just a headache, time, and it actually costs more for the pleasure. Free Google Workspaces killed 90% of people self-hosting, too bad it costs tons of money these days.
Both perspectives may be entirely correct, but the devil is in the details. It seems more right to say that in general, self-hosting email is hard, rather than act like it's easy because you figured it out and everyone else running into issues is just inexperienced/dumb/not good enough. Chances are, those folks were dealing with more complexity than you have.
Plus I have as much experience between commercially and personally, and that is my take. The fact you couldn't give one single example should give people pause.
Dovecot and the postfix submission port (587) are only accessible internally, or through my home wireguard VPN.
Rspamd catches just about all junk mail. I might have one or two messages a week slip through. Moving messages to/from the Junk folder trains Rspamd to recognize spam/ham.
Make sure the IP of your mail server has reverse DNS in place, and set up SPF/DMARC records and DKIM signing, and you should be fine. I've been doing this for a decade and never had any problems.
I do maintain a separate, paid email account at a commercial provider for things like banking. In case I die, I don't want my poor wife to deal with my crazy email setup.
This topic always brings up so many hysterical naysayers, I almost wonder if some are paid Gmail shills!
The first year was rough because I didn't know what I was doing. Mail would not go to or from certain domains with no indication of why.
Once I finally figured out all the security settings, it's fine. My domain will talk to anyone (including Google) and has been for several years.
My private email is my primary account. It's on a domain with my name, so I use it everywhere.
Is it worth it? Absolutely not. I wouldn't recommend you run your own server unless you just want the experience. It's far, far easier and cheaper to just buy the domain name and use the inbox your registrar offers. Or a paid service to host email under your custom domain.
For me, I have other users on this service, and the cost and risk of migrating them to a new service is way, way too much. I'd have to decrypt their inboxes and merge them into the new service, and I just don't want to go there, so I keep this server ticking over until we all die I guess
I have not had trouble sending to Gmail. On important emails to Gmail addresses I BCC my own Gmail account. If my account does not get the mail I figure there is a problem. For me it always goes through.
My setup is stable. In the past ten years I had my current setup, but also other domains hosted via Google Mail, or by my hosting providers etc. They all had migrations, discontinuations etc. I migrate them all to my mail setup. I have no such hassles with the email servers I myself set up. It does not have large changes (by servers I mean the primary and queuing mail servers).
I have a lot of experience doing this though, not sure how hard it is for others, but it is a lot easier than the old days of configuring Sendmail with m4.
That said, I used to host my mail server for a while and my mail always went to Gmail inboxes. My recipients don't use some exotic providers like outlook or yahoo, so it was pretty smoooth for me. I know that people have issues with self-hosting mail, so may be it was just luck for me.
My opinion is that as long as you correctly configured stuff on your side, it boils down to IP history and reputation. So buy VPS from some expensive provider in a reputable country (so spammers are unlikely to host in this provider), ensure that received IP is clean (and its subnet is clean), set up DNS, keep it running for few months and it should be good.
Maddy instead of Postfix because Postfix configuration gets quite messy with complex virtual users logic. I was trying to refactor it, trying to devise some sort of DSL to make things more bearable - and found myself wishing there'd be something more straightforward. Maddy was a good fit (and it has a modern easy-to-understand codebase that I can hack on it if I need to). It's much smaller and simpler than Haraka, but still has all the features I need, and looks secure (I swept through the source code and found nothing obvious).
Dovecot is a very fine piece of software. I was considering Wildduck, but haven't figured out a contingency plan if I'm going to dislike it in the long run. As long as all my mail is a Maildir (with some extra indexes I can just ditch), I have no worries - I'll always be able to read it. So I've just set up dsync for the failover (in case one of my servers has a network or power outage) and can't be happier.
Once (somewhere in early 2010s) I had a backscatter issue that put me on a couple DNSBLs, fixed that the same hour I've learned of the issue, filled out some forms on RBLs I was listed on, got removed in a couple days.
Delivery-wise, I've had some issues with Outlook (to an extent I've almost set up an "embassy" there, tricking the system into thinking that my domain is hosted with them and using their own MSA for my outgoing mail to Outlook-hosted domains - proof-of-concept had worked, but I haven't really bothered to implement it in practice as they had suddenly started to receive my mail without issues, and this whole idea is a dirty hack), but otherwise email delivers fine. A couple times Gmail put my emails into spam, but mostly everything worked as expected. Not that I send many emails, maybe a couple dozen a year - it's mostly receiving.
I'd say the main problem is lack of decent mail clients.
Rainloop for the web, but I almost never use it - it’s a backup option for me, I’m mostly hosting it for friends and family.
It also hosts my private git repositories in /var/git and a few other services and websites.
One day, I will migrate it to Arch Linux.
OTOH, Debian exim config is the incarnation of evil.
This has evolved since 1996 (!)
I still have issues receiving emails. Especially from financial institutions. (Wells Fargo emails just... stopped. So I gave up and setup a gmail just for them.)
I have issues sending emails from my custom domain.
IRL I always have to explain that yes, my email address is YourCompany@MyDomain.com, but no, I don't actually work for YourCompany.
Over two decades, I have played whack a mole with a number of problems. Switching to a hosting provider that does not allow me to forward <any>@customdomain.com helped. It only allows explicitly forwarded email accounts, and setting up the email address explicitly passes most email validation tests, and I suspect the underlying IP is less likely to be blacklisted by fin tech that is extra careful about who they email.
But they had fixed it since then, now emails work fine. I suspect they had some bug on their side, because I haven’t seen anything suspicious on mine (and nothing had changed on my side).
Idk what was that.
About a third just point to my name servers and accept email for my domains and most of the popular email domains to give spammers a feeling of accomplishment and to pre-populate ISP DNS caches. Those servers just dump the email into a single flat file. I use these for sending to known malicious entities or when I need a throw-away temporary email address. I just grep out what I need from the text file.
The remainder of my domains just point to non-routable public addresses.
In-bound is easy with DNS checks filtering all spam. Out-bound is somewhat out of my control, so I use a relay like smtp2go or sendgrid for reliable delivery.
I mostly use aerc but recent versions of Roundcube are pretty slick as well.
If I wasn’t trying to trim every possible cost currently, I would still selfhost. It’s trivial if you have any sysadmin/syseng experience and a little time for setup. IME, the main thing that trips people up has already been mentioned. You need SPF, PTR, and DKIM records to get delivery to big providers. You can usually request to be removed from blacklists if your IP was flagged before you got it.
I did this too (migrating from Google Workspace using imapsync¹) and it works great. Most people don't realize that Apple has been hosting email for decades and is quite good at it.
¹ https://blah.cloud/miscellaneous/migrating-google-workspaces...
Disclaimer: I'm a passionate Tutanota customer.
The server-under-the-stairs (dl380g7, Proxmox running containers with services, one of those being the mail container on which Debian/Exim/dovecot/greylistd/spam assassin/dovecot-sieve). Backup to several external locations, mail archived for about 25 years.
The setup I’m using is:
- I have a mylastname.com domain. For serious accounts like my bank and insurance, I use companyname@accounts.mylastname.com. - I also have a myrandomdomain.com. For most other online accounts I register with companyname@accounts.myrandomdomain.com. - I have been playing with a sieve script to automatically create folders and categorise incoming mail for each address. It needs a couple of small tweaks still that I didn’t quite iron out before going away for a few days but should be pretty straightforward. - In limited cases where I want more anonymity (eg. Facebook, Reddit, now Twitter), I create a masked email and set up a filter to put it into a folder myself. - I use a different address to log in to my account instead of myfirstname@mylastname.com
For me this is a reasonable mix of security and convenience. Depending on your level of paranoia you could be slightly less paranoid (company@yourdomain.com or company@something.yourdomain.com), or more paranoid (use masked emails for everything - there is now integration for this in 1password so if you’re using that for a password manager it’s very easy to manage. I love the automatic categorisation using my own generated addresses gives though.
Self-hosting email isn't hard if you know the basics, and there are lots of resources online. Not like, say, self-hosting asterisk, which is a bit more of a pain (and I also do, reluctantly, but I barely use it, webrtc+asterisk is useful).
Having access to mail logs has been very helpful for various things. I also have aliases that connect to various (self-hosted) Gitlab projects for work, where we use service-desk.
I know that for most companies, self-hosting does not make sense financially, but our company does not make a ton of money, my time is fairly cheap (a few hours a year), and having control on the infra means we don't have to worry about how many seats/licenses we have, we can just do whatever we want.
Quite often, working with other companies, we end up having these artificial barriers because they can't afford to create an account for me on their issue tracker. So now we have a dozen companies using our systems, because it's all setup, it just works, and no artificial barriers, just a bit of disk usage and good backups.
Tech wise, in the old days it was sendmail and pine/mutt/etc, more recently courier, these days postfix and dovecot.
I wouldn't ever consider outsourcing something as crucial as email as it's not e2ee and being able to diagnose delivery issues and fix them quickly is very important.
Also related: former mailop
Used to be on a Legacy Google Apps for Domains before they pulled their forced changed to paid accounts.
Migrating to postfix at Hetzner that forwards at home hosted server is not for the faint at heart. Still struggling with the plethora of settings that have to line up with dns and forwarding.
For testing your email settings, internet.nl/test-mail is highly recommended.
I've set up smtp servers for various projects with SPF DKIM etcaetera and they work (even Google accepts them) so it can still be done.
One day I'll do it for my own server ;)
But honestly, without an easy solution like yunohost I wouldn't attempt to selfhosting personal mail even I already know how to do on different platforms.
I have no interest in managing my own email server though.
Postfix + dovecot + rspamd on Debian on my own colocated hardware. Have been doing it for over a decade at this point (probably over 15 years)…
Has moved service providers a few times, as I try to stay on small providers with a high level of clue :)
Is it an utter PITA to have to chase a dragon's tail of stupid things that you don't control to have RELIABLE e-mail? Yes.
While running your own server is great for a hobby or as a learning exercise, 99%+ of people just want e-mail to WORK. And many of the big providers have an endless amount of hoops to jump through, and have little to no transparency or feedback when things aren't working.
I use one of these domain+email only webhosters, doesn't cost much more than the domain itself and they do a better job than I did in my spare time. I still get spam but it's maybe 5 mails per day and I have catchall on a 20 year old domain.
I can only receive mail, but having my own domain is useful because now I can move without telling everyone a new address.
For my websites, I use Zoho's free mail plan + nodemailer for sending automated password reset emails and the like. It works pretty well.
I can't go back to anything that doesn't have a sieve implementation for filtering.
Incoming messages are sent directly to my SMTP server.
Outgoing messages are sent through the ISP's SMTP server. (This service is included with the internet service from the ISP, so I do not need to register for this service separately.)
The IP address rarely changes, but sometimes it does, and when it does change, I can update the DNS records.
* self-hosted on a rented physical server-system running debian gnu-linux (stable)
* setup via ansible-roles
* using my own DNS-setup with isc-bind
* (open)ldap-directory (qmail-schema)
* exim4-ldap with my own configuration (not the distro-standard)
* spamassassin + clamav
* courier-imap
* roundcube for web-access
* gnu mailman ver 2.x
* letsencrypt for certificates
just my 0.02€
ps.: i'm doing this since the 2nd half of the 1990ties ... so i really know what i'm doing :))
Stories of people having domains unjustly taken from them are rare, especially on "traditional" registries like .com/.net/.org
Most registrars can auto-renew so as long as your card stays valid you’re fine.
Yes?
> Do you run a mailserver/host email accounts on your domain?
g-suite free - multiple domains + simplelogin (previously used anonaddy) with a custom domain (actually a subdomain of one of my gsuite domains)
To make it do what I want, the mail configuration is a bit complicated. Well it is for me, doing it once every 2+ years and forgetting most of how it all works in the meantime. Presumably it's not complicated compared to large-scale setups or for people who do this as their main job.
Upgrading the OS every 2+ years is a pain, because there tend to be quite a few configuration file changes in the new OS, sometimes different software packages, I have to review everything well enough to understand it, then merge my own changes from the previous setup. And in fact the last time I tried the upgrade failed anyway, I don't know why. Very possibly nothing to do with my mail setup though.
So I've concluded that for OS upgrades it's probably as quick to install on a new VM and reconfigure everything from scratch. That takes me me about a day in total, working from my bad notes, comparing the old and new systems, copying data files etc. Which does not seem like a good way to do things, but it does mean I end up with a clean installation. Also I've been able to switch the underlying VM, so now I'm on the allegedly faster/cheaper ARM instead of Intel architecture. That would not have been possible otherwise, because the OS and application software binaries are different for the different architecture. Also the underlying SSDs are the newest types, and the latest ones are allegedly faster/cheaper.
I have seriously considered trying to script the setup or updates somehow. But I think there are enough changes in the OS between upgrades that for a single system it's just not worthwhile. I'd have to review and fix the scripts, then run them once, then not use them for another 2+ years. Plus learn the tool (Ansible?). Plus I would guess some of my configuration changes are not common so I'd probably be writing custom stuff for the tool as well.
And... I almost never have problems with it. As far as I know mail is sent and received reliably, and for years the server almost never went down. More recently it did hang a few times, though I don't know why. But I suspect it was because the wimpy VM just gets overloaded sometimes. It's not easy to find the reason though, and I didn't want to spend much time on it. Also since that happened I've upgraded from Ubuntu 20.04, and for whatever reasons, it's been fine so far.
Whether this is all worthwhile or not I'm not really sure nowadays. Originally for me it was to learn about the mail software, also because some software I wrote for an older project did a lot with email, so I had a kind of professional interest. Also I wanted to use my own domains and be independent of an ISP. Plus I sometimes use the VM for other things, such as a rarely used OpenVPN server in the past, replaced with Wireguard now. Also technical experiments sometimes. And if I could think of anything worthwhile to have on a website I would use it for that.
But if you just wanted easy and cheap email with your own domain(s), I think the best way could be a "traditional" hosting service. Compared to what I'm doing, you could get far better performance, storage space, bandwidth, backups, reliability, maybe security, everything else. And probably more cheaply, with near zero time and effort and hassle, and without having to be try to be a biennial mail configuration expert. Plus if there were any problems the hosting company would fix them, or provide support for you.
So for most people I think it's difficult to justify running your own mail server. There'd have to be some specific reason. Which might just be that you want to!
In 2019 I migrated from the vServer to a dedicated setup switching from Debian to OpenBSD, from Postfix to OpenSMTPD. I use Dovecot, the only thing I ever tinkered with were spamd or spamassassin. The only things to do were DKIM, DMARC and SPF.
I have never had an issue with mails not arriving on either end, I rely on it personally and professionally.
I am very much not an expert in the area, nor really interested. I just set it up and it works. Should some emergency thing happen I'll just put on my last backup. But no data loss has ever occurred in these years. Just use your RAID setup.
So far it has been more reliable than Gmail (which was down, had bugs, etc., just search Hacker News ;))
I think there is a huge amount of FUD in the area. E-Mail itself is very reliable and handles any issues very well. Since it's a super old tech nothing much changes, just DKIM and so on. I think a lot of tech people are all to used to hosting stuff that constantly changes which makes them very scared of it. But e-mail you just read into, set it up, add a couple of comments for your future self and might easily outlive you.
Of course you should do your updates and of course I wouldn't recommend to do it like me and do that as a whim as a teenager with no experience, but also it worked out.
I'd recommend OpenBSD because the system is sane, you get OpenSMTPD which is compared to other extremely easy to set up correctly. It doesn't have all the cruft that others accumulated.
Here some things that you need to not forget so others won't think you are a spammer:
* Set up (and test) SPF
* Set up (and test) DKIM
* Set up (and test) DMARC - even if it's just the record part
DON'T FORGET TO SET UP the PTR record. I do that for all systems, but somehow even with people that should know this things from their job people seem not to.
Read up on each of these things to get it once, configure it properly, add comments, maybe write down some notes somewhere, have backups. And you are good to go.
If you are scared, you can just do all of the above and still use what you currently use. Spend some times (years?) to see if it works for you with non-important stuff. If it does, you can switch stuff over. If not you learned something.
This is not me saying that I think you should do it, but it's an option if you are interested. You can just get a cheap server and try it. In the very worst case you'll have learned something and made your own opinion.
Also on IP reputation. I think there's something off here.
You go to say Mailchimp/Mandrill/Mailgun/etc., because of their "reputation" and a month later you notice that their reputation sucks, so what happens is they upsell you to get your dedicated IP (own thing, more expensive package, etc.).
I never had a problem with IP reputation but I'd assume that your hosting company would give you a new one if you brought it up.
IP-reputation is largely a thing of the past.
IP addresses are ephemeral and email providers understand that. Especially with IPv6, where an individual can have access to tens of thousands of addresses. So, all large email service providers have spam filters based on content, and domain reputation, not IP reputation. Misbehaving hosts may be periodically blocked based on IP, but this is never permanent. Otherwise email providers would be blocking the entire IPv4 internet by now.
If you have a domain that does not spam, and it is using DMARC + DKIM, then there is sufficient proof that an email sent on behalf of the domain is authentic and the email will be accepted no problem.
The only real problem with hosting your own email, is that it is usually impossible to host it from your residential internet connection, as ISPs do not allow you to 'own' your IP-address, by not allowing you to create PTR records. This is all on purpose though, ISPs block it and email providers require it. It's is because there are just so many residential IP addresses that are part of a botnet sending spam.
I'm lucky enough that my ISP provides me with a fixed public ipv4.
I use postfix as an smtp server and dovecot as an imap server (pop3 is disabled). I run them on Red Hat Enterprise Linux via a developer subscription.
So here's the deal:
1) learning postfix, dovecot and general email stuff does require some time, but essentially it's an one-time effort, email protocols do not change that much over time. you might split the effort over time and configure bits part by part. needless to say, the bit to learn as soon as possible is how not to be an open relay (that is, not to relay spam).
2) deliverability is the main problem. many services (mostly gmail) blindly assume that you're a spammer because you're sending from a residential ip address block. this is a form of discrimination in my opinion. needless to say, no matter if you setup spf and dkim, gmail is still going to deliver your emails to spam folder.
personal pet peeve: whenever i open my gmail inbox (used to do that due to having an android-based phone) it was full of promo emails that google willingly delivered to my inbox. yet my legitimate email are delivered to spam folder. i hate the gmail team passionately.
3) i have an mx backup via a virtual machine on aws, the smallest cheapest instance available. it was a quick fix when power went out at home and i was on vacation, but it's been worth keeping around for the last ~3 years.
4) maintenance is effectively a non issue. i just go through the configuration file when updating the base operating system to make sure new versions of postfix and dovecot still accept my configurations
5) server-side filtering via sieve is just awesome. when i was ~16 i used to reconfigure my filters in thunderbird every time i reinstalled my laptop, nowadays my emails are always delivered by dovecot to the correct folder (i'm using maildir to store mailboxes)
6) you do need to take care of backups. but mailboxes are essentially text files, they do not require special care and their compress ratio is very good. i keep my emails on a snapshottable filesystem (zfs), and that comes in handy when doing maintenance (worst case scenario I rollback).
7) for incoming email, when you set graylisting, spf policy verification and dkim verification, 99% of the spam is rejected by the server. i actually get most of my spam from my mx backup host, which was configured in a hurry. but i add an "x-from-mxbacup: yes" header when pulling that mail from there via formail and do filtering on that via sieve in dovecot. works okay.
8) antispam servers are clunky, i decided not to run one and just reject mail on the basis of greylisting, spf and other checks. it works okay.
9) after a while i gave up with deliverability and started using amazon ses as a relayhost. Nowadays i've got a small dedicated server with a public ipv4 and a configurable reverse-ptr dns. I should be looking into that again, but nowadays i'm fairly busy.
In general I'm very satisfied.
i recently moved to microsoft o365 basic, which is good and also gives the smtp etc, so you get any client to work. The biggest pro i miss is a catch all mail solution, which was amazing for protonmail. i didnt have enough time to tinker with o365 to find out catch all mail so far, everything else it beats protonmail hands down.
https://docker-mailserver.github.io/docker-mailserver/edge/e...
I don't have a web UI for it at all (though I briefly integrated with NextCloud Mail for that) since nowadays I just use Thunderbird or a similar client for that. Personally, the aspect that I like the most is their approach to doing some common actions, in the form of their setup script, you don't even have to connect to the container to use it: https://docker-mailserver.github.io/docker-mailserver/v11.3/...
The aspect that I like the least? Well, just how mail servers are architected in general, they even have a nice page on this: https://docker-mailserver.github.io/docker-mailserver/v11.3/... (things get even more complicated once you introduce additional security solutions into the mix, like ClamAV, SpamAssassin, or others). Projects like docker-mailserver or Mail-in-a-Box abstract some of that complexity away from you which is good as someone who doesn't want to sink hundreds of hours into it...
However, it's still there in those packaged and preconfigured/integrated components, rather than as one monolithic package that does most of the stuff you want, if you would compare some of the web servers out there against how many of the mail servers work, for example. Then again, something like Apache2 has bunches of modules anyways, so maybe that comparison isn't as cut and dry. Regardless, it feels like even with the scripts and the documentation, it's just a time bomb that's waiting for X years to go off when the pieces will decide to no longer play nicely with one another, and then you'll really need to dig into it, like when your Linux distro bootloader decides to die one day.
Outside of walled gardens, that might be one of the reasons to do a double take before putting everything on your new self-hosted mail server. Test drive it first for a few months/years, gradually move stuff over (and be sure to have tested backups that you can actually restore/access when necessary), maybe look into recovery addresses in other providers where applicable or where it makes sense and so on. If your mail client backs everything up locally, the server going down shouldn't be the end of the world and technically you could just move over to a new instance if everything's FUBAR, but having to drop everything because you cannot receive any new e-mails (such as a link to confirm doing something) would be rather disruptive in certain circumstances.
It really pisses me off.