So happy to see that. Hopefully more people will run their own E-mail instead of being slaves to the large adtech "free" e-mail providers. We need more balance on the Internet.
If your ISP provides you with an e-mail setup that you can use with a conventional mail client where you enter IMAP4 and SMTP credentials, chances are you can use that for SMTP sending. I.e. from the perspective of sending mail, your ISP can't tell that you're a server; it thinks it's just Outlook or Thunderbird connecting to it.
Receiving mail is no problem; your ISP just must not be blocking port 25.
It's handy to give yourself mobile access. When I send mail from my phone, it connects to port 537 of my own mail server which provides authenticated SMTP over TLS. It forwards to the aforementioned ISP. (I can't connect directly to my home ISP's SMTP server from my phone because the phone is on a mobile network unrelated to that ISP; the ISP's SMTP forwarding servers are firewalled so only the subscriber addresses can talk to them.)
Use your email provider's SMTP, even if it's you yourself.
Almost all cloud providers with dynamic-load ephemeral IPs will show up on ban lists eventually due to vulnerability scanners, bad spiders, and spam/voip drops. However, it is far more common for Spamhaus free tiers to quietly go sideways when no one is looking.
Gmail/Outlook have their own peer policies that serve their own business posture. Google does require administrators register in their clown system as a user to exchange email, but it is effective policy that adds nuisance cost to people spinning up 30 servers a day to spam people.
Firewall Rate-limits are effective on small single-domain servers. A modern email server in Go that is isolated from each user space greatly simplifies the possible setups. =3
I registered for their Postmaster Tools, which says
No data to display at this time. Please come back later.
Postmaster Tools requires that your domain satisfies certain conditions before
data is visible for this chart.
Refer to the help page for more details.
Outlook was even worse, and I just told my Outlook users to change providers.
Eventually I capitulated and got Google Workspace, and now everything gets delivered perfectly.
Worked for a company self hosting famous brand emails. They would get blocked too. Imagine telling the band manager of a famous classic rock band that their email to their label was being rejected due to being black listed for spam.. (cc’ing the managers team)
Stop fooling yourself, it does not work fine. If it did you would not rely on that google outlook or yahoo account
Personal/private/family email can be easily self-hosted. You just need to know a few things to get it set up properly.
At 25+ years of hosting email through multiple hosting providers, this has been my experience multiple times. To be fair, happening less often with DKIM et al, but those are relatively new inventions.
Getting a dedicated server with an ISP that does a decent job at keeping their IP blocks clean for email is about the best you can expect. Setup the appropriate SPF/DKIM/DMARC and get along. There's really not too much more to be done these days. Even the big guys don't always get along.
Sorry, but that's FUDish. The reality is if you do a proper email setup (DKIM, Reverse IP, etc.) you will be fine.
If you happen to actually get an IP that somewhat recently happened to be an email server, that was also sent out spam, which isn't something that's likely at all then you'll notice very quickly (just send an email to to gmail, etc.) and what you'll do then is tell your hosting provider you'd like another IP address, because it's not fit for your purposes.
I've been running, moving, switching IPs, providers, domains since 2005 and still am and there is just SO MUCH FUD. It's not hard. It's a one time thing. Personally I never ran into IP reputation issues ever. These are email addresses used in a professional capacity (B2B, communication with governments, etc.) as well as private use ones.
Pretty much every ISP, every university, etc. runs their own email server. Many companies do. Many private people do.
I have run them on the side for those 20 years now, partly as a hobby and so far the uptime was higher than Gmail's and since I use them for private, professional and sometimes for government communication I am dogfooding it and I would have very much noticed if anything bounced.
I have gotten bounces when a setup was initially broken, like when I do something like sending a test email to Gmail and that was off.
The reality is that IP and domain reputation aren't really great ways to filter spam anyways. Yes, it adds, but what makes you think that nobody sends emails from Gmail, a university or other stuff? What makes you think that spammers use static domains, etc.
Heck, not even DKIM and SPF are any guarantee. People will spam you from servers with extremely good reputation. Looking at my spam box most of them are from situations where accounts obviously simply haven't been blocked yet.
No serious spam filtering is done with IPs or domains being an "all or nothing" thing.
Also it's a two-way street. If a user of some email provider doesn't get their email and it becomes known people will be wary of it. And nobody expects the email landscape to stay static. There are newsletter and transactional email services all over the place, lots of marketing platforms running their own email servers and so on.
It's not like everyone does something magic, nor does everyone have connections, money or time to talk to all these companies. An email service not accepting emails won't exist for long.
And something that's also important to realize: If you do start using a transactional email service they oftentimes will make you pay EXTRA for a custom IP so you DO NOT share it with others, so you get BETTER reputation than the cheap one. And you configure your own domain with it. So why wouldn't these emails get delivered? And many of those don't run their own data centers and not all of them have their own IP blocks (though some have).
It's just if you couldn't even do regular private emailing, emails would not be the thing every website uses for login and communication.
> The reality is if you do a proper email setup (DKIM, Reverse IP, etc.) you will be fine.
You're not getting reverse DNS on a dynamic home IP.
> Pretty much every ISP, every university, etc. runs their own email server.
Yes? And what did I say: if your ISP has mail servers for you, it can simplify things greatly if you use them.
Smaller providers will generally not black hole legitimate message like Hotmail does. They have (paying) customers awaiting those messages. Junk folder? Sure, that can happen sometimes.
> You're not getting reverse DNS on a dynamic home IP.
I don't think anyone here is suggesting running a mail server on a dynamic IP.
> And what did I say: if your ISP has mail servers for you, it can simplify things greatly if you use them.
Only if you want to be using their domain and if you're not sending (too many) automated messages.
> Only if you want to be using their domain
No, that's simply not how SMTP routing works.
So you want some third party provider to be delivering mail on behalf of a domain for which they don't have even have the basics like DKIM and SPF set up, and hope for better deliverability than you could easily obtain with your own server?
Your SPF record, created by you, indicates that the certain forwarding servers you have chosen are authorized to deliver mail for your domain.
When you change SMTP providers, you update that.
E.g. a year ago I switched from Shaw to Novus (two Canadian service providers). I edited my server's SMTP credentials to the new Novus server and user ID, password and changed the SPF record to bless Novus servers as being my delivery agents. That's it; mail was flowing through thew new configuration.
The ISP doesn't know anything about my domain or any of its DNS records.
Yes, they have better deliverability than I could obtain with my own server directly, because my server is on a dynamic subscriber IP which makes it a pariah in the world of mail delivery. Sending from it directly to mail exchangers world over is a nonstarter.
I could pay for some server in a cloud data center somewhere. What for? I have no issues with mail delivery.
DKIM though.. ?
But I get your point: it might beat a home server on a dynamic IP on deliverability. Both options seem troublesome.
In my SPF record I have novus.ca.
So I don't care what IP addresses Novus's mail servers use, as long as they identify as <host>.novus.ca.
DKIM-signed messages can pass through SMTP hops. I'm not briefed up on the details of DKIM, but to my best current understanding, the originating domain signs the body and certain headers (not all of them) with its private key. When the message passes through multiple SMTP hops, some headers get added, like "Received: ...". I believe, these headers do not invalidate the DKIM signature. The relays just cannot be messing with the body of the e-mail, Subject:, From:, Date: and such. SMTP relay is not like a mailing list repost.
I'm now looking at some raw e-mails with DKIM signatures. It looks as if the signatures plainly specify the names of headers that are included in the signature, via a field that starts with h=, listing colon-separated header names.
Now suppose you contact that server and complain about being rejected.
Wouldn't it be ironic if they respond like this: "We receive e-mails from gmail just fine; fix the problem yourself, or use gmail".
This is how self-hosted e-mail people throw each other under a bus and let gmail win, while pretending to hold self-hosting as a cherished value.
(They would most likely be right about having to fix the problem yourself, unless they imposed some locally authored and highly unreasonable/dichkeadish filtering rule. The superfluous rhetoric about gmail would be almost as obnoxious as their rule, though.)
If your client uses MS for email and doesn’t receive your invoices, it becomes a big deal.
https://news.ycombinator.com/item?id=35691618
In the end I set up a gmail account just to route all my outgoing mail through, with a whitelist of specific servers I know won't reject me for no reason (i.e. a few very small email services or friends who also self-host). Defeats half of the purpose but what can you do? There's nothing else I can possibly do to make my emails reach hotmail inboxes - I've exhausted all of their phony support channels and advice articles and clearly they just want me to go away and stop self-hosting.
I send mail from several domains out of my mail server. The PTR record for that host actually doesn't match any of the forward hostnames.
It does mean that you are slightly less than perfectly self-hosted, in some sense.
If your mail server is in a position that it can send mail directly to any mail exchanger in the world, rather than going through a forwarding host, there is the advantage in that it can use end-to-end TLS.
One of the server sends and receives emails for the forum, sometimes up to 1000 messages a day. It was set up 5 years ago.
Maybe this is a serious issue when you use popular VPS providers/IP ranges, but I use smaller providers, and just don't remember any email-related issues everybody are talking about.
For me, email self-hosting as easy as installing mail-in-a-box (for sending+receiving) or just plain exim/postfix (for sending only), with proper configuration.
We also have receipt tracking, which isn't perfect, but shows a >93% open rate.
We did have an issue delivering to a specific provider, but that was resolved by updating our DKIM with a more robust key length.
https://www.xmox.nl/faq/#hdr-won-t-the-big-email-providers-b...
Won't the big email providers block my email?
It is a common misconception that it is impossible to run your own email server nowadays. The claim is that the handful big email providers will simply block your email. However, you can run your own email server just fine, and your email will be accepted, provided you are doing it right.
If your email is rejected, it is often because your IP address has a bad email sending reputation. Email servers often use IP blocklists to reject email networks with a bad email sending reputation. These blocklists often work at the level of whole network ranges. So if you try to run an email server from a hosting provider with a bad reputation (which happens if they don't monitor their network or don't act on abuse/spam reports), your IP too will have a bad reputation and other mail servers (both large and small) may reject messages coming from you. During the quickstart, mox checks if your IPs are on a few often-used blocklists. It's typically not a good idea to host an email server on the cheapest or largest cloud providers: They often don't spend the resources necessary for a good reputation, or they simply block all outgoing SMTP traffic. It's better to look for a technically-focused local provider. They too may initially block outgoing SMTP connections on new machines to prevent spam from their networks. But they will either automatically open up outgoing SMTP traffic after a cool down period (e.g. 24 hours), or after you've contacted their support.
After you get past the IP blocklist checks, email servers use many more signals to determine if your email message could be spam and should be rejected. Mox helps you set up a system that doesn't trigger most of the technical signals (e.g. with SPF/DKIM/DMARC). But there are more signals, for example: Sending to a mail server or address for the first time. Sending from a newly registered domain (especially if you're sending automated messages, and if you send more messages after previous messages were rejected), domains that existed for a few weeks to a month are treated more friendly. Sending messages with content that resembles known spam messages.
Should your email be rejected, you will typically get an error message during the SMTP transaction that explains why. In the case of big email providers the error message often has instructions on how to prove to them you are a legitimate sender.
When I say I'm self-hosting, I mean I have a machine under a table right here in my home: True Scotsman's cotsman's self-hosting.
I drop SMTP connections from servers that simply do not have matching forward and reverse DNS. This rule eliminates like 90% of spam. It's a good rule and I won't make any exceptions. There's no way to contact me. Your bounce message tells you what you have to do: get your DNS ducks in a row.
That's assuming your residential ISP even bothers to assign a generic PTR record to your IP.
Even that doesn't work all the time. hotmail is currently bouncing emails from me[0] even though Microsoft's own sender reputation thing[1] says my IP is in good standing.
[0] with a link to [1] just to rub it in.
Residential IPs are spammy, so if for some reason you've decided you're going to let SpamAssassin to handle them post-delivery, it would make sense to give them a high score.
If the residential IP is in the MX record for the domain, even more so if the domain passes DKIM, why not?
However, if the host passes this check, and all other tests such that we decide to accept the mail for delivery (to be further processed by SpamAssassin), at that point why would we want to apply any score in SpamAssassin regarding the residential IP. We already decided to pass it.
Big providers often only support their own forms and ignore open sources trust providers.
Small providers often do not maintain their email services which will simply auto spam your mail/domain, when it does not come from the big 10 providers.
I set up a mail server with NixOS 5 times in a row with 5 different Hetzner Cloud IPs and each of them arrived fine at Google.
Also works for Microsoft services?
> Also works for Microsoft services?
No, Microsoft seems to have an allow-list for servers, and ban everything by default (the laziest of approaches).
But that's OK: Because Microsoft does it so poorly, Microsoft email users check their Spam folder all the time anyway.
* web.de
* gmx.net
I also have to say that I always used a hetzner root server. Moved multiple times due to an upgrade.
I ALWAYS had to manually apply for removal of my Webserver. It worked for Yahoo. At that time it did not work for Gmail and Microsoft. I no longer was blocked but if I was writing for the first time to a recipient, I landed in the spam folder.
The software I used mailinabox and mailcow. Both had self checks. All green. I also used external scanners to check my config, all fine. You can check my GitHub (razemio). I even contributed to some issues for mailinabox.
This is not only true for selfhosting but also small providers. As an example:
* mailbox.org (auto spam Gmail and microsoft 2018)
All of this was a long time ago. Maybe I am just depressed from the bad experience and the FAQs are telling the truth. However it is hard to believe for me.
I am selfhosting since 1997 and I am working in programming / DevOps.
> It is a common misconception that it is impossible to run your own email server
... the FAQ then goes on to give all the reasons that argue it's really really hard and probably not worth it for most people.
Same.
> No issues.
Many issues.
I haven't had any problem in that regard in over 20 years of running a mail server on an old PC, on residential ISP connections. SPF, DKIM and rDNS config seemed to keep all the big players happy.
Which just made me realise I don't even have valid rDNS anymore, but it still works.
I beg to disagree, as I've been running my own E-mail and sending from my own IP address for [checks notes] the last 25 years or so.
- What happens when one of the big cloud providers arbitrarily start putting your emails in spam?
Are there solutions to this? It feels like the biggest value provided by "big email" are these two things
For your second point, you live with it. I haven’t found a solution, at least. I’ve never landed in spam for corporate offerings (cloud O365, google workspace or whatever they call it now) or (very rare these days) anyone self-hosting with rspamd or equivalent, just regular personal mail (hotmail, gmail, iCloud, etc). That’s usually pretty easy to detect and work around (“hey I sent you an email” “oh I didn’t get it” “did you check your junk?”) Irritating, but not the end of the world.
I’m going to try hosting from my residential IP sometime this year, now that I have sufficient redundancy in terms of power and networking. I don’t know if I’ll have better or worse luck than with hosting providers’ IP ranges, though.
Mail server still gets blocked by random domains. Nope. Done with hosting email. Everyone assumes you are spam and won’t accept your mail unless you pay them (to be your mail provider).
i spent some time today buying a new domain and setting up mox on a hetzner vm. the IP was on 3 blacklists on first check, after fixing the reverse dns it's on 2, one of which is apparently fake? dkim and dmark seem to be working, sending a mail to protonmail succeeds the checks, and yet it lands in spam - however, i'm confident once the domain is older than "just now" and i've set up DNSSEC (takes 1-3 days for this to start working in my country apparently) things will improve.
worst case i'll have to request a blocklist to unblock me, but i'll see.
Mixing email with the drive service in the account is actively hostile.