Self hosted email continues to steeply decline
labs.ripe.net
labs.ripe.net
(Blog author) runs the email infrastructure measurement project at Live Direct Marketing
Google offers "mail merge in Gmail to send personalized email campaigns, newsletters, and announcements to a wide audience." [1]
Based on how much @gmail.com spam I used get within GMail I can only assume they treat mail from their internal accounts much more favourably then us dirty self hosters.
minimum effort business model posting minimum effort slop to promote minimum effort agenda. the world disgusts me. RIPE really should know better, this isn't a mistake.
edit: fuck me, from the author page:
Email: art-online-shop@yandex.ru Website: https://check.live-direct-marketing.online/email-stats/
- may as well be a spam email itself
Don't listen to the naysayers.
* I am still using the same domains that I've always hosted mail for * Since I am sending from new IP addresses, but want to retain the ability to send from the old MTA until it is decomissioned, the new MTA uses a different HELO identifier, and the dependencies for that are all in place (forward and reverse DNS addresses and updated SPF records) * The new DNS records are still in the same administrative domain as the old ones
If you set up a new VPS for sendmail email, the first thing you should do is look up the IP address you have been assigned to assess it's reputation. If it's bad, you may be able to be assigned a new IP through some providers, but you can always just spin up a new VPS, check it's IP, and if it's clean, use it, otherwise drop the VPS, create a new one, and check the IP.
Another option is to buy your own CIDR range. The smallest subnet you can buy is a /24, but your addresses will be portable between ISPs giving you complete control over your IP reputation, and the ability to warm up IPs for sending when an existing IP you are sending from starts to get a negative reputation.
Always make sure you set your rDNS or PTR record to match the A record for the mail server's FQDN.
Set up opendkim, opendmarc, and policy-spf or equivalent, make sure your DNS is all aligned properly, and set your DMARC policy to enforce. You can start off enforcing on a small percentage of outgoing email, and if that looks good, set to reject 100% of DMARC failures.
My stack is postfix and dovecot with opendkim, opendmarc, policy-spf, with postgres to hold domains, mailboxes, aliases, etc and some simple scripts to get postfix and dovecot to use the postgres db. I used postgres for reasons not directly related to sending email, so the same thing can be accomplished using something like SQLite or even flat text files. I ended up writing a perl program years ago to manage the whole stack from a single cli command.
I'm not suggesting my stack is anything special. I use the daemons I use because I have been using them for years and understand them very well. If you want something more than just a TUI, there are a lot of options. I still use spamassassin, but I know there are newer alternatives.
The main driver behind how I chose the stack I use is knowing exactly how every part of it works.
Running your own email server without understanding how all of it works is what people mean when they say that you should not host your own email server. It's not enough to copy/paste from a tutorial, because you will run into issues in which something isn't working right, but since you just copy/pasted you have no idea how to fix it.
I worked for an online publishing company that sent a mixture of quality content to double-opt-in subscribers and transactional emails, and moving traffic around on your IPs if you have more than one to maintain reputation. We sent over 500 million emails a month using a combination of self-hosting our MTAs, and also using ESPs in some scenarios in which even the best IP/domain reputation practices aren't enough to get inboxed. Always set up feedback loops and maintain your lists if you are sending newsletters. Set google postmaster tools.
One of the reasons I run my own mail server is that I create aliases any time I sign up for an account on a website that are unique to that site. github@example.com would be an alias that forwards to an actual mailbox that is unknown to the rest of the world. This way I can tell who has sold my email address, or had their mailing list compromised, and I can simply remove the alias to stop any spam going to the real inbox. Doing that can reveal some pretty interesting things about how your information is bought and sold by data brokers.
It seems plausible that domainnames registered to individuals hosting their own email would _not_ likely be in the "top domains"
Perhaps this headline could be more accurate if it was something like
"Self hosted email among the top 1 million domains continues to steeply decline"
I hate LLM writing, but the economics checks out.
That's the thing, good writing is expensive, massively so, much more than people think.
Letting the machine do the writing however shows us that many people have no idea why writing is expensive, nor do they respect that perhaps it needs to be expensive. Communication is incredibly hard, very few master it. I'd even go so far as to say that clear, precise and readable communication is so hard that we do not have sufficient training data to build an LLM that can do it.
Letting a person write article, like this one, doesn't necessarily improve the quality, but it does show us that some thought and effort have at least been put into it. There was a cost and you deemed that cost worth it for conveying your message. Without the cost there's nothing stopping you from wasting peoples time.
I stopped reading at this point. I cannot emphasize how much I HATE this AI writing style.
Is this in the same vein as using — em dashes?
Are people with minimal regard to typography and style doomed to lower their standards because the masses can't even?
It's kind of sad. I compare it to when Memes started becoming popular as communication tools on the internet (mid 2000s from what i recall): Instead of writing text to convey the idea they wanted, people just pasted a meme (my face when ... whatever).
Now with AI, everyone write everything in the same way.
I'm looking at this as having the potential to destroy one of the most harmful technologies to have been invented: social media.
If it gets sloppy enough, people may revert to more humane communications.
I stopped here, anytime an author (or likely AI) feels the need to tell me they're being honest or "genuine" I don't need to read any further.
I'll assume honesty but the minute you explicitly tell me, you've probably got an angle that I'm too dumb to see, but I'm not dumb enough to keep reading.
Everyone should aim to be this lazy. Everyone is trying to maximize their time. This is what software has always done. It helps people be lazy in some respect, and then, so far, we have always found new stuff to do with the time we freed up.
LLMs got decent at writing, and before long they will be better at write-ups than the average human writer and we won't look back.
Of course, this is contingent on your current goal: If you want to get better at writing, you should do the writing, just like you want to do the math, if you want to learn multiplication. Writing is fine as a tool just to get information across. It needn't be more than that for everyone.
Delegating this task to automation serves to undermine all the masters and indeed strengthen the servants.
Short term this offload is good for people; long term it’ll lead to mental rot where people are hard pressed to put their ideas forward and give up in exasperation when they don’t have the assistance of their servants. Their ability to even make the proper requests to the servants may deteriorate enough that their existence may come to rely on the benevolence of their servants.
Judging by people's response, that doesn't seem to be the case.
> and before long they will be better at write-ups than the average human writer and we won't look back.
I'll read articles written by them if and when that happens, but not before.
When you start digging in you realize that (like most things) there are multiple different definitions. LLMs are very good at putting together a lot of words that are spell correctly with grammatically correct sentences (better than me - though I'm particularly bad at this). However people often want to add other things like "correct" (the facts are right...), or some notion of not using too many words. LLMs tend to not be as good at those.
I think if this post had not gone the AI-generated-text-route, some people here would easily have considered it interesting for its content, even if it was entirely AI generated.
That only applies when the article's author just prompted an LLM (and did not supply the LLM with any data you did not have) and then posted the results.
I'd expect most article authors who use LLMs are more involved then that, checking the results and correcting (or telling the LLM to correct) thing that it got wrong.
If the article is from someone who knows more about the subject then you do then that will be more valuable that what you would get asking the LLM yourself (even if the prompt used in the article didn't include anything that your LLM didn't know).
Kill me. With a spoon. Slowly.
"Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting."
LLM-written content is now too common to be interesting and complaining about it does not make it stop, it just derails comment sections. The reptitive complaints are getting as annoying as the LLM-written content!
Personally, I usually check the comments before deciding to read something so I appreciate that others have saved me the trouble of reading this article in particular. If the author doesn't care enough to use human time to write it then I am not going to use human time to read it.
Likening this to restaurant reviews and publishing houses is disingenuous. One is nothing like this and the other is not helpful in that it’s such a generalization of influence as to be not useful. Everything is an influence doesn’t mean anything.
Presumptuous.
Why should your opinion be worth more than those who would like to retain a semblance of meaning in what we use to convey ideas? LLM writing is sheer trash. It comes off like a coke-addled car salesman who is making a pitch to VCs about his latest scientific discovery that will 200x their investment.
Poorly written LLM content is NOT a tangential annoyance. If this is too much, maybe you should simulate HN using
Generative Agents: Interactive Simulacra of Human Behavior https://arxiv.org/abs/2304.03442
Because it's like trying to hold back a tidal wave with a few sandbags.
Maybe people in their 20s and early 30s have enough energy for this and everything else that life demands, and I applaud their effort. But I have nothing left to give after this industry has already taken so much.
I LOVE reading LLM content. I find that they tend to present things in a way that keeps my attention - something I have struggled with my whole life. I also use LLMs to assist with managing my own thoughts and helping make things coherent overall.
Sure, there is a LOT of slop out there, but the promotion of quality content is what crowd-sourced curation like HN tends towards. Much of the LLM assisted content here has a high signal to noise ratio, and is clearly a lot more involved than simply prompting "write an article on X" - despite what the haters say.
And generally I find that the content isn't that high quality. You're getting someone who superficially appears like an expert and looks like they've thought through the problem, but if you know anything about the problem, you'll find that the articles are usually weak (Gell-Mann Amnesia).
That sounds like a you problem. Develop the mental discipline to reduce your response to negative thoughts and you'll lead a more content life.
The only way around this seems to be hosting with Google, Microsoft, or maybe Proton. Any other ideas?
Email spam has made self hosting untenable.
If I ever go back to self-hosting email, I would do the same except I would maintain a whitelist of friendly mail servers to which I would send directly. Or maybe just relay emails to Google/Microsoft via SES and send the rest directly.
How do you know? I have seen blocks of ip addrs blocklisted from AWS due to others abusing nearby ip addrs
I'd argue that self-hosting is the first step. The second step is helping everyone you know get their own domain and host their email for them. If they don't trust you to not read their email on your server, point them to somebody else who offers mail hosting.
The more people we can convince the internet is something anyone can help build outside of the silos, the more likely it is to survive.
Also, set up seed accounts on different email providers and subscribe them to all of your lists, this helps you get an idea for what other people are seeing when they get the emails. If it's going to a seed account at gmail, and getting sent to the spam folder, you can try tweaking the email and work around the issue with some A/B testing.
I think techniques like HashCash by Adam Back should be revisited.
I'm still refusing to give up the good fight.
Haven't had an issue since I moved to 365. Costs a little more but so much less hassle. I don't like it but I also need people to actually receive my emails.
Have yet to face deliverability issue with this setup (Gmail, outlook)
You should self-host, so you can do the experiment for yourself. You might be surprised. You can always stop self-hosting if you don't like it.
For some time it was possible to just send a special request to the provider asking them to unblock the port, but this is becoming less common, for often they simply deny the request.
Users who have been hosting their own email for many years have often been grandfathered in so they might not even be aware of this.
They should also deal with the individual accounts etc, but thats the difficult problem.
Ironic given the amount of crypto spam that exists.
But I self-host email for ~20 years, and I have not managed to get rid of Spam without not also getting false positives.
I first used SpamAssistant, in later years Rspamd, but I feel like they are just not good enough. Also I find Rspamd config pretty incomprehensible.
Hosted email like also does not solve this, e.g. GMail filters way much (e.g. important company correspondence leading to orders almost being lost because they landed in our GMail spam, so I had to turn off the spam filter entirely).
Of course this assumption is fully invalidated by open-access mail service providers: maintaining communication with xXx_kitty_slayer_xXx@tepidmail.com tells absolutely nothing about sincerity and authenticity of messages from DrJohnSmith@tepidmail.com. Not only there's roughly zero correlation, it could be argued that correlation is slightly negative if one does not expect out-of-the-blue contact attempts. Funnily enough, with the critical mass of non-spam migrating to providers, it becomes sort of positive signal, fueling feedback loop: even if the majority of spam came from gmail, messages from gmail would still be more likely to be non-spam than messages from rncrosoft.com, because most non-spam comes from there too.
On top of that, there's a lot of soft-spam. Companies will happily send out critical contract-related information and unwanted-unasked advertising from the very same domain and even the same address. The big providers are not shying away from selling shielding services to spam agencies.
That "someone" could - in theory - require pgpgpg signatures attached to messages and have origin-based filtering/ranking, but good luck convincing people to set up their gmail/outlook to even generate the signature, let alone publish it reliably/revocably. At the receiver end, your only reasonable bet is filtering/ranking the content. It's probably not a strictly uphill battle, but you definitely need significant amount of volume and tagging before your models become good enough and it stops to the case.
In reality, even if it sounds a bit ironic given above, the best route to fixing this is self hosting, especially in the cases where your messages are needed. The more people try to send mail from their self-hosted instances and insisted that it was the problem of the other end to complain to their service providers if they do not receive the messages, the less hegemony the cartel providers would have. And maybe we could have some sort of client auth for email.
It would be better if we came up with new protocols that solve the email messaging problems and then gradually wean ourselves off email rather than trying to "fix" email.
In a sense this is already happening although instead of migrating to open protocols which solve the email problems we're migrating a lot of messaging to walled gardens which solve the problems (e.g. WhatsApp).
I had hoped that something useful would come out of blockchain technology that would allow for a better messaging protocol that was spam resistant, e2e encrypted and decentralized by design but I didnt see anything.
I completely agree with this.
And setting them up properly is not even that difficult nowadays
I self-hosted for years and I did all the standard things you're supposed to do, but I had tremendous problems with Gmail and Outlook. Other providers usually went through without issue.
FWIW, in my case, I am running the server on Hetzner, even though is supposed to have a bad reputation, but I have been using its IP for more than 10 years for other things (web server, personal services etc) so I guess that helps
Unfortunately this only talks about what DMARC policies are published by domains. I was hoping they will go into some analysis of how many mailboxes actually hard-enforce DMARC rules on ingress (as in, respect the DMARC policy as published by the sender's domain, p=reject or whatever else it might be)
I have been hesitant to enforce DMARC rules on my server. Partly because maintaining it was a pain (see opendmarc - maybe better with rspamd now), but also because on my test runs I saw plenty of legitimate incoming mail that simply would get rejected if I were to enforce DMARC on ingress.
I suspect big players like gmail have some kind of soft enforcement/reputation/machine learning approach to DMARC, but this is way too much work for a small time admin like me.
The only thing I think can compete with that is some kind of decentralized solution, also something I've thought about a bit. Like a Keyoxide-based[0] - or similar - setup that makes it virtually impossible for one to lose their online identity (since lost keys can be rotated out once one has - access to - enough proofs), and then that identity can be used - without fear - to route associated data, etc. Of course adoption of what could become an adjacent or new standard would be the next hurdle to overcome...
On the other side of the coin, what would happen if google bans you? That's certainly not unheard of.
I have been running Postfix for 22 years, from a big corporate installation to my own on a Raspberry Pi Zero. It just does its job.
I've self-hosted for about 2 months, and promptly switched to iCloud. Just because I've been spending too much time calling people and telling them to check their spam folders.
Recently our business moved to another domain. We had a guy who has spent over a month "warming up" traffic, so our new domain won't be tagged as spam by default.
I've done it for decades, but I don't know much beyond the basics of email hosting tbh, however my understanding is that these days email coming from self-hosted servers carries a much higher spam/junk risk and so often automatically marked as spam.
Maybe there are ways to avoid this happening, but when a few clients all reported that they were not receiving my emails I didn't have much of a choice but to switch.
However you do have to live with the occasional blacklist when you host on a VPS like I do. Some idiot in your IP range may get blacklisted for spam and then you get caught in it. It’s generally temporary.
And there are some corporate policies that will flat out reject emails from you which is fun.
I’ve probably spent less than 10 hours over the last 5 or 6 years on maintenance.
>And there are some corporate policies that will flat out reject emails from you which is fun.
This is what keeps me away from self-hosting it, though. As long as there's some possibility that my ability to deliver email could get nuked, all because someone else made a mistake, it's not worth it to me, especially since using / sending email is mandatory for doing most things. I guess I'm just not really willing to take that sort of risk
Or you can pay to get off the list.
Not all email services use blacklists anyway. Bit of a racket.
You can mitigate against it by hosting on a dedicated IP.
But I understand the risk. It’s really annoying when I send someone an email and they call me some days later wondering why I hadn’t sent them an email. Failures are often silent on the sending side.
It’s not for everyone. There are risks.
I've written about it five years ago, still applies: https://schumacher.sh/2021/05/10/running-a-private-mail-serv...
The problem lies with the big e-mail providers marking your e-mail as spam, even if you have everything configured right (MX, SPF, DKIM, DMARC, etc.) and you're not on any blacklists. They use their mysterious opaque algo to determine that your e-mail is spam and you have no idea what you could do to not get in peoples junk boxes.
And that’s not hard?
Running a modern mail server, a spam filter, a self-hosted webmail, and learning just enough about DNS and networking to get mail through: this is what’s not that hard.
Figuring out how to stay whitelisted, or accept ending up in people’s filters, asking them to whitelist you while your public IP gradually gains trust, is a job.
I just wish these big parties would give you a more formal way to make clear you're not a spammer.
There is only one metric of an honest mailserver and that is time spent not blacklisted.
This is true "proof of work" on behalf of the mail server admin.
Running a mailserver on an IP range easily accessible to spammers (e.g. a cloud provider) gives you a negative start. You need to invest in "good IP addresses". I will gladly pay money to anyone who wants to do this. There are medium-sized players who deliver affordable, top-notch service. Turns out $5/mo. is enough for human service, because something that just works doesn't need a lot of it.
I think gmail can just gradually bounce self-hosted email and people will give up and move to their service.
My self-hosted email filters too little spam, while GMail etc filter too much (e.g. important company correspondence leading to orders almost being lost because they landed in our GMail spam).
Edit: typo
I used Gmail for Domains for a hot minute, but never really liked it very much. Soon I found Fastmail, and I've been using them ever since (so about 13 years now).
Also had a couple of ui pages on our app to have full email functionality.
This also means custom integration into our existing ecosystem.
EDIT: "I see no upsides to X" means "X is NOT good". (Responses suggest that this was not clear.)
If your email is Google or Microsoft, there are other people with self-hosted email who feel pressured to use their Google or Microsoft accounts to send email to you.
TBH, I blame IT leadership. The easy choice is Google or Microsoft. "No one ever got fired for buying IBM" is now "No one ever got fired for subbing to Office." Someone needs to bite the bullet and stitch together an alternative array of services. They've cultivated their own monoculture by hitting the easy-button and forking over for Office365, etc.
Which if we're being honest, Microsoft and Google products are rarely best-in-class at any one thing that they do. Teams, SharePoint, Google Docs, someone somewhere is doing those things much better. Those competitors just don't necessarily have a whole ecosystem that makes it a big Easy Button for corporate IT to smash. And they sometimes charge as much by themselves as Office365 costs in whole. But I know one problem my own employers IT team struggles with a bit is putting out the constant fire of different teams deciding the Microsoft tool we pay for sucks and going rogue and trying to subscribe to something else. Dropbox was a big one, but Teams and Planner alternatives come up a lot too.
Not pretending to have all the answers but it doesn't seem to be a completely intractable problem.
I remember when Gmail came around and everyone was more than happy to hand off their email to Google. No spam and ability to send and receive huge (for the time) attachments I think were the main things people wanted.
Not sure if that was before or after everyone was happy to hand off their web browser to Google.
NO, and I avoid both of 'em as much as I can. (Note clarifying edit to my initial comment above.)
"PRISM is a code name for a program under which the United States National Security Agency (NSA) collects internet communications from various U.S. internet companies. The program is also known by the SIGAD US-984XN. PRISM collects stored internet communications based on demands made to internet companies such as Google and Apple under Section 702 of the FISA Amendments Act of 2008 to turn over any data that match court-approved search terms."
https://en.wikipedia.org/wiki/PRISM_program
"Section 702 permits the Attorney General and the Director of National Intelligence to jointly authorize targeting of non-US persons reasonably believed to be located outside the United States.
By targeting, it is meant that US persons or persons located in the United States may not be the intended targets of the collection. The targeting must have as its object the acquisition of foreign intelligence information, as this is defined by FISA."
And yet, a quick search of "psychologists near me" in Google Maps gives me a list of professionals most of whose email addresses have MX records pointing to Google servers.
It was complicated to get security, DNSSEC, DKIM, DMARC right this way. I did it just under a year ago, around September 15, 2025.
I tried to follow and understand what I was doing, and keep good records on it. It was a complicated process, but everything was up and working at the end and I received email my server sent me. It felt good to be able use this simple script to send myself email:
#!/usr/bin/env python3
import smtplib
from email.message import EmailMessage
msg = EmailMessage()
msg["From"] = "(user's from address)"
msg["To"] = "(my gmail address)"
msg["Subject"] = "(a test subject)"
msg.set_content("(a test body)")
with smtplib.SMTP("127.0.0.1", 25, timeout=30) as smtp:
smtp.send_message(msg)
print("Message submitted successfully to local Postfix.")
I understand that as my server's reputation grows, it becomes very powerful to be able to email anyone without any intermediary except potential spam filters. I am very careful about what I put on my server.
Nevertheless, I had a recent experience that was a little sad for me. I wanted to forward some mail to my gmail address automatically, and used ChatGPT Sol 5.6 on medium thinking time to help me with the administration to do this. (I find extra high and pro thinking times are excessively long.)
I was very disappointed with the results, which included a very badly written regex for authentication. I'll write up my experience in an experience report. It was a big disappointment for me.
The burden on administrating one's own email is already very large, and it is disappointing that we can't really rely on frontier models to help with it.