Running your own email is increasingly an artisanal choice, not a practical one
utcc.utoronto.ca
utcc.utoronto.ca
- Getting your own domain and using a provider such as fastmail or proton is a first step that gives you lots of security fom arbitrary. Because you own the doorstep, you can change provider without having to inform all your contacts of the move. You're also more secure from unilateral moves from your provider.
-Hosting your own mail server means that you are responsible for the persistence of your mail. It's a nice artisanal thing to do, and you may be satisfied to know that no one is reading your mail.
-Sending your mail yourself is the real hard part, because you need a stable IP that is accepted as a legitimate mail sender. Moreover, you need to monitor this property in the long term. Every mail server has their own way to choose who is a legitimate mail sender, and it's an ongoing pain to check that.
You're not forced to go all the way, you can simply pick a domain to secure the frontdoor, or you can host your mail server without sending mail by yourself, etc. You can also self-host, and change your mind later without much impact.
I personally would incite everyone to do at least #1 for safety reasons, #2 if you want to fiddle with the system to know how it works, and to avoid #3.
It's still using Google but it allows plenty of control and management, and I can take my domain anywhere with minimal stress. It's a decent compromise.
Zoho has a $1/yr plan.
I’d add another thing:
- Hosting your own mail client. You can self-host roundcube/mutt/thunderbird/or even an imap server that just fetches (and possibly deletes) email from the remote server using something like mbsync. This mail client/server doesn’t need to interact with any other mail server apart from the mail provider that receives the incoming email, be that gmail or fastmail. While paid ProtonMail can be used for this, it’s a bit of a hassle with their lack of native imap support.
This also shouldn't be a problem most of the time if your email server supports TLS; Google currently sees 81% outbound email encryption[0], so you can imagine roughly 4/5ths of email servers support it.
0: https://transparencyreport.google.com/safer-email/overview?h...
It's possible to punt on this by using SES for outbound, while continuing to handle inbound a different way. Obviously SES doesn't count as fully self-hosted, but it does solve (or at least significantly ameliorate -- zero issues here) the reputation problem
How does this work? Do you just sign up for aws, then set your outbound SMTP to whatever SES provides?
This would be something that provides in a single package an SMTP server, an IMAP and POP server, pre-trained spam filtering, and maybe a web server with a web-based email client, and a simple setup program that asks a few basic questions such as your domain name and configures everything on your end and provides help for configuring things elsewhere (such as with DNS, such as telling you what to put in your SPF and DKIM and DMARC records).
This is meant as something to handle your mail during the time it takes you to find another provider. It is meant to be something you can quickly install on a VM somewhere, point your MX record at it, get a Let's Encrypt or similar certificate for it, and not be losing mail while you are between providers.
It should have a quick start guide that includes details on signing up and getting a Linux VM up at major inexpensive VM hosting places. Amazon Lightsail, Hetzner, and such.
It should make minimal assumptions about your Linux environment. Probably it should not use the SMTP, POP, and IMAP servers that are packaged by your Linux distributor. It should use minimal servers that are written specifically for the emergency mail kit.
It gives me all the flexibility I need with almost no work or maintenance.
There are enough mail providers that I could easily switch to that I don’t need a piece of software. Switching from gmail to yahoo, proton, apple, outlook, or juno is a simple domain adjustment and has me back receiving mail within the TTL period.
I could go through the process again (not fun) with some ridiculously long .com/.net or other OG tld which are probably somewhat more resistant to rent seeking practices like this or I just suck it up and hope it remains pricey but not egregious.
EDIT: .ca is not particularly tech community related, but that doesn’t matter to me.
One thing I’m not sure of is what happens if I want to switch registrars in that time—will the full 10 years of ownership transfer to the new registrar?
If you switch registrars, your domain validity continues as before. Your registration information is with the registry for your TLD, not with the registrar alone. So your 10 years of (future) ownership will carry over. Two caveats to note. The first is that you can’t transfer a domain within 60 days of purchase or renewal. The second that I’ve seen is that a transfer to another registrar requires a one year renewal for the domain. So I’m guessing you may not be able to transfer a domain that’s already at 10 years (even taking into account 60 days after the renewal to keep it at 10 years). But if your domain is at nine years, you’d surely be able to transfer, and it would become 10 years at the time of transfer.
The expiry date never changes through all these actions.
My understanding is that SPF makes forwarding like this no longer possible if the original sender's address is to be preserved.
Forwarding, on the other hand, made it possible to do a simple redirect of one address to another (eg. ~/.forward) but strict SPF rules will deny the forwarder as a valid source for the mails.
The goal is to get my personal domain to be my email domain for exactly this reason.
For my soon-to-be-born son, I'm registering him a personal domain immediately, and I'll turn it over to him when he's old enough to have email - save him some trouble.
I have my own domain since when I was about 15 years old and used that for a while on a digital ocean droplets. It's incredibly easy to set-up.
The only issue I had was that other people were not getting my mail and I sometimes it was not even reaching their spam folders. Probably because Google/Microsoft were blocking that IP range of Digital Oceans.
Nowadays I just pay for a personal Gsuite license and use Google Infrastructure.
Much simpler that way and I'm almost guaranteed that my mail will reach the recipients. You only need to set-up your DMARC / SPF records and point your MX records to the ones that Google provides.
If you put an address or domain in the safesenders list; they do literally nothing. Like you can just totally spoof the domain entirely.
However if you use transport rules as per their rec, there’s all sorts of stuff that will still get flagged, and you have to to reference ATP, anti-phishing, anti-spam policies. Much of which aren’t even in the Exchange admin panel, rather they are in “security” and buried in hamburger menus galore.
And what’s best. They don’t even have any documentation for how these modules interact or what order mail is processed in. I had a case open for months thst finally got escalated to someone that was able to explain the issues we had with specific list serves/domains getting flagged.
In the end my only option was to whitelist emails classes as phishing and route them to junk rather than keeping them in quarantine. Even though it was a 99% accuracy rate sans this single domain.
The guy was really only able to commiserate with me. We are but a number and not a big enough one to get Ms to change a thing. Their best recommendation was to deploy an edge device like proofpoint/proofpoint hosted and just handle it from there.
I get what they want to do. They are trying to make the crazy email RFCs easy for devops guys thst don’t give a damn about how e-mail works. But it’s still hard to keep up with as they constantly just move stuff around and change their own standards on a near monthly basis.
So even if its not listed on the domains MX record but you can suss out they are an office365 tenant receiving mail, you may be able to relay off it and spoof to high heavens (especially if the edge device reccomends you....ahem...whitelist your own domain and not use transport rules). In fact especially if you can do this.
For example i think MS forced proofpoint to change their config recommendations as an outcome.[1]
from the page on [1]:
"Due to major complaints, Proofpoint has opted to change change to the format of ensuring Proofpoint mail is not scored via the O365 system. This rule will allow external email to come in still, but will follow O365 scoring. This is to ensure no mail is lost."
[1] https://web.archive.org/web/20200807173336/https://help.proo...
No, having quality spam and fraud filtering, and quality security, that you host yourself, is by far the much harder problem. I would argue that outsourcing your email to Proton or Tutanota is not running your own "artisanal" email server. By the way, even with those email providers, I still have terrible spam and fraud emails getting through filters that I never would have seen with my GMail.
I just don't see what people see in gmail part from the google brand - which surely isn't a good thing anymore.
Most people I know? It is almost a daily occurrence. Including if senders are in address books, and "not spam" is clicked when found in the spam folder.
Most people I know see legit emails in the spam folder, all the time.
So far I've been lucky to rarely receive actual spam, but I've often missed out on important emails too often.
My experience is similar, I receive much less spam on my actual mail than I do on gmail.
That's not a claim I'm making. The goal of my message was merely to help people see that there are several steps you can take from using a gmail account to relying on no third party. When I talk to people, they often don't realize this, and especially, how easy it is to set up #1.
> No, having quality spam and fraud filtering, and quality security, that you host yourself, is by far the much harder problem.
Handling spam is not an easy problem, but it's one where you have all the cards to take actions. On the other hand, having your mail properly delivered is something where people have wildly different outcomes, and for people with bad outcomes it's "impossibly hard, and there's no action you can take about it, unless you personally know the right people at the right places".
As a bonus: I get to see report which of my emails were classified as spam or not opened.
Also, first 100 mails per day are free (which has been enough for me so far).
I am running my server for 15 years and couldn't be more happy with "artisan" infrastructure.
I don't want or use webmail (sluggish), I don't want others fingerpoking my emails, I don't want various compulsory registration systems (like requiring my phone number out of """security""" reasons like google), giving others the ability to kill my account and do me a huge amount of work and on the top of that, it breaks the sites registration schemes, I have set up a script that accepts any email with some special structure and each and every registration gets a specially customized mail address (that I can calculate in my head, no configuration needed) that can be resolved back to the registration.
Getting spam? I am sick of you, whatever? No issue, just REJECT the whole address. It is used by only one site, like smart people don't reuse passwords, I don't reuse email addresses.
And you would be surprised, how many sites sell email addresses to others, and I know it as every one gets its own email address.
Rspamd eats the spam just as good as "ai infrastructure" /s
Even if you go for 3rd party email infrastructure, registering a domain is a must, so you can switch the provider fast if it gets vampirized.
Out of my whole infrastructure (100% self hosted, as said, for 15 years, actually more but not 100%), the email server is the part that needs the least attention.
The response to the author would be: nice that large providers have webmails and some other quirks that I don't want or need. Feel free to use them, but I have freedom.
postfix. dovecot. rspamd.
I started using that with Fast mail, they call that Masked address. Best spam filter ever.
I haven't administered e-mail servers for 20 years, but back when I did, this started to be a problem that eventually became insurmountable. I used to manage a small business oriented ISP. We were multi-homed with a /18 that we used for everything. I had a customer that was a reasonably sized organization that dealt with tourism and conventions for a major city. On one of their websites, (which we hosted with IPs that came out of the same /18 as their mail server,) they had a directory of vendors who were associated with them. ONE of those members had a website that had been hacked/defaced. This got our entire /18 on a blac-khole list. They had an employee that was trying to send e-mail to someone on a system that was using this black-hole server to filter spam.
When we explained to them what the problem was, we got glassy-eyed stares back at us and a, "just fix it." I told them that, they would need to remove the link to their partner's site from their website in order to get them AND all of our other customers using numbers inside our /18 de-listed from this particular black-hole. They asked, "We have hundreds of partners who pay for membership in our organization and being listed on our website is one of the benefits. How can we possibly police every one of those websites every day to make sure there's no defacement or serving of any problematic material from any URL in any of those domains?" That's a decent argument in my opinion. And I tried to explain that different black-holes have different policies and no black-hole is demanding that anyone use their system for filtering. I tired contacting the organization that was using that black-hole to explain the situation to them, but they weren't interested in discussing it. As far as they were concerned it was our problem to deal with.
This kind of problem happened dozens of times with varying degrees of severity but with increasing regularity and it was one of the primary reasons we quit hosting e-mail and started re-selling another vendor's solution. That was a long time ago, and maybe black-hole lists aren't a thing anymore.
> If so, were you able to successfully communicate with and/or reasonably work through whatever issue got you black-holed?
Yes. Practically all black-lists have a de-list form that one can use, and most seem to auto-delist fairly fast as soon they don't get any more reports from honey-trap and other sources.
We do have a few custom written ways to detect hacked accounts, and we don't allow users to set their own passwords. We also tend to discourage/deny users who do newsletters and other "higher risk" form of email. All emails sent by websites is sent through different servers, which also mean that a hacked website does not impact the reputation of the email servers.
Events with black lists maybe occur once a year and as I mentioned above, fixed fairly fast. One good tip is to keep an automated eye on the mail queue and react quickly when things start to look wrong.
There are a few sites where you can plug in an IP address to see if it's on any blacklists. A handy thing to do before setting up a new server is to work with your provider to find a clean IP address beforehand. Here's one that I have used: https://mxtoolbox.com/blacklists.aspx
Now I have incentive, this is annoying. Maybe I need a static IP as well.
Yes. Twice.
In the first case, the mail provider was our ISP; and they got themselves in some mainstream blacklists. The problems getting that sorted out were part of the motivation for bringing mail in-house.
In the second case, there was some academic departmental mailserver and they were using some list incorrectly; using an extremely-opinionated list to block when it should at best be used to score.
This wasn't in itself a big deal, but one of my boss's correspondents was a senior professor in this department and they had some important business; and the postmaster was a dick, and wouldn't help. Boss didn't want to use some secondary email address; I had to show him how to set up an alias on some commercial server, which was second-best, but he was in a hurry.
Boss was angry with me and barked at me. If you run a mailserver for some group, one you assembled yourself, then people expect you to take responsibility for sorting out any mail problems. Well, they're right: you have taken on that responsibility. You made it, and you're running it: who else can they complain to?
[Edit] My point is that it's not hard to set up an artisan mail system; what's hard is that you create a job for yourself that is at the same time networking, user-facing, and technical. It's an interesting learning point, and I recommend it. But don't underestimate what you are taking on.
Assuming logic applies to humans is painfully wrong. I wish it wasn’t.
This. So much this.
I will happily run my "artisinal" mail system for myself. Would I put customers on it? Oh, hell, no.
I, sadly, always recommend that companies pay money to Microsoft for email. You are really paying for the customer support service rather than the email service.
There are plenty of other email providers which are worth considering, and I'm sure some of them have half-decent customer support.
You would, sadly, be wrong.
Microsoft customer support is "least bad" among the email providers.
That is a massive indictment of email providers, but it is what it is.
Customers still have problems occasionally _sending_ email to one domain, which is over 15 years old and sends <1 email per day. If they initiate and email us we can't send them a reply (if they're on MS email, sometimes). We use an outlook.com email nowadays as a relay and have to treat MS using customers differently still despite using a relatively large supplier.
Some years ago, I was lead to believe, you could pay a third-party to add you to what was effectively MS's whitelist.
Aside: back then I was doing some webdev and supporting IE5+ so I already hated MS about as much as one could.
Never paying to enable interoperability that is part of being a reasonable web citizen/company. Paying them just reinforces the negative behaviour.
There is also a guy, Jar, who runs a rather his own email service, mxroute, quite successfully. Users love it and he seems to know his stuff.
They probably work well on quite limited hardware. But I guess it largely depends on traffic / on the number of hosted users.
Approximately nothing. I run all my email infrastructure on the smallest available $5/mo Linode and it's way overprovisioned even so. I'd take a smaller VM if they offered one.
So much this. I've actually contacted companies to tell them they've been compromised because I started getting phishing emails. I quit after the third time of reporting it and being told "we haven't been hacked, someone in your friends group has and you just can't read email headers".. right because someone in my friends group emails "mylocalgym.com@mypersonaldomain.com" to schedule group activities.. then six to twelve months later I get an email from HIBP telling me said website was hacked and my email was compromised.
I tend to sign up for a lot of things (I'm seeing over 150 unique email addresses I receive emails from using this scheme), but I guess I'm just getting lucky.
Also, just out of curiosity, where does one sell email addresses, and how much are they worth? I take signups on a few websites, and I'd never sell my users' email, but I'm just curious to learn more.
I don't know how to monetize said system but it would produce both social and economic value.
When I'm opening an account at Example Bank which uses example.com domain, I avoid creating dedicated mailbox or alias with words "example" and "bank". exmplbnk@, xmplbnk1234@ or similar seems to have better deliverability when I'm attempting to contact the other side.
Likely to help cut down on phishing.
This might sound like gatekeeping, and maybe it is. When these systems were designed, they were not designed to be used by everyone. They were not designed to be commodities that are bought and sold, with the most valuable trinket available being the attention of the user. But this is where we are.
Few are capable of running their own _anything_ on the internet, and even fewer have the desire to do it, because if you run it well for yourself (as an individual), someone else will want you to do it for them because you are already doing it, so it's not that much more work, right? \s
Decentralization limits monetization of anything, so that is going to be a non-starter for investment of resources. Unless you are trying to have your infrastructure survive a nuclear war, no one is going to provide the means to build anything big unless you can sell it or the users of it.
The notion that anything really works on the internet with the assumptions that were made in the 70s and 80s, and the realization that what holds most of it together is the blood and sweat of ops, duck tape, and fever dreams consistently astonishes me. In the not so distant past, someone paid me to write them a custom FTP server. In the 21st century. It's like being asked to whittle an engine block out of a tree.
I'll go further: centralised systems can emulate decentralised systems, but not vice versa. Thus, ultimately, the only USP of a decentralised system is that it is decentralised for the sake of being decentralised, and nobody cares much about that. Centralisation is inevitable, and wins out every time.
People certainly care enough about centralization once it's consistently abused in ways that hurt them (which always happens eventually, given enough time). Our existing anti-monopoly laws came about like that.
Plus I don't see much evidence people care. This argument reminds me of Accelerationism, which doesn't seem to work either.
Everybody know you show never even read YouTube comments. Posting them is just insane…
Serious question: what if you're on a network that blocks non-HTTP[S] traffic? How would you read your mail? This is a problem I hear of quite regularly.
Im happy to pay for my own cellular data and vpn to avoid networks like that. Including tethering my laptop if needed.
There are also correspondences where you are only the receiver. For example, when you order things online. Gmail doesn't need to know what you are shopping for.
It's a very, very impressive edifice that's been created for identifying and tracking pretty much everybody irrespective of their direct, immediate interaction with the entity doing the tracking. I honestly think it's kind of funny that such a potentially insidious system was manifest for something as principally vapid as ad targeting.
Option 2 in particular is super appealing.
I’ve tried a bunch of pricacy-focused email services and have been let down by one or more aspects of their service. Pretty much all of them managed to handle sending Ronny satisfaction, though.
So setting up inbound to run on my own gear and paying a couple bucks a month for others to deal with dkim and and domain keys and all that other crap… that’s brilliant.
Thanks for the idea!
(Or rather given everything I've read about self-hosting email, not regardless, this is my preference...)
The author mentions quality in big email service but only passingly mentions what that encompasses. Smooth, responsive, well-worn, ceaselessly preened, and smoothed-over end-user UIs are important. Unfortunately, the open-source alternatives are comparatively rough.
(As a long-time developer and more recent designer, I write a lot of open-source code myself. I understand that these are complex and tedious problems to solve. However, without frank critique, "Open-Source Alternatives" will always be "Alternatives.")
Every interface I saw needed fundamental design work. My recent research showed 2+ decade old interface layouts w/new features just bolted on, visually complex toolbars, menus, and lists, little editing for views and controls, and comparatively unattractive designs(, which even if it doesn't matter to you, that doesn't invalidate its importance to others.) Even this crowd— people accustomed to configuring complex applications— lament the clunky interfaces.
To me, most open-source interfaces are like eating on a diet. Your sense of accomplishment offsets the discomfort... at least for a while. End-users, however, don't have or need, that holistic view of the service. To them, the interface IS the service. DIY/tech accomplishments are abstract and indirect factors, at most. For most, it's like eating on-diet, but someone else loses weight. Attractive alternatives make that unsustainable.
So the real hard part isn't technical— it's assembling an email stack where users don't feel deprived for having chosen it.
The solution is more collaboration between design and development expertise within the FOSS. If you have a position of authority in any FOSS projects, I implore you to be open-minded when presented with interface design ideas.
Happy to talk about productive ways to engage with designers and design feedback.
Because there aren't open-source contributing UX/UI designers. Almost all open source interfaces are quick work done by mostly backend developers.
Secondly, that few contribute as designers rather than developers is definitely a chicken and egg situation. Designers time and effort is universally seen as less valuable than developers’ and therefore more readily dismissed or minimized. People are worse at taking critique for things they’re not confident in, and as you note, most open source projects are maintained by developers. Ever give a brand new developer a code review? Yeah. That’s about what it’s like critiquing an open source project’s beloved “quirky” interface.
I’ve seen eager designers post issues in repos— some with complete wireframes and rationale having done a good amount of work already, asking for specific types of feedback— only for their system to be instantly bikeshedded into oblivion rather than productively discussed. Unsolicited contributions are often viewed as superfluous expenditures of dev time, or even viewed with outright suspicion or hostility. If it’s not submitted in the form of bite-sized PRs ready for production with the understanding that existing devs can veto any changes without any real justification. Going from a haphazardly assembled UI to a properly designed UI requires fundamental change, and that’s a lot of work. Would you contribute code in a project with those competing requirements?
Before any of that, any designer interested in open source software has almost certainly made the mistake of griping about the interface for gimp, or git. It’s a good preview for what lies ahead.
I pay $12 a year for email hosting, $10 a year for the domain. I use name.com and I presume (though I have not tested) that if I needed a human to talk to, I would have much better luck than with Google. I also don't have to worry about a snarky Youtube comment getting me locked out of Youtube, Youtube TV, Gmail, GDrive and everything else.
If you are going this route for security purposes make sure they have proper policies and are not a susceptible to social engineering.
At least Cloudflare is offering such thing, but it’s enterprise option [1]. I would assume many others have similar offerings as well.
[1] https://www.cloudflare.com/en-gb/products/registrar/custom-d...
I have come to suspect new MX servers are spam-holed by default until enough people click “Not Spam”, which is an absurd hurdle for a single user hobby server.
Yes, a fresh (or: previously sending spam) IP requires some warmup time until providers like GMail will let you anywhere near the inbox.
And if you're not sending out a high enough volume of emails, no chance.
I monitored all the blacklists, filed ownership attestations with receiving domains, the whole nine yards. It’s sad that a microscopic MX server can’t be default trust instead of default spam for the first two messages a week.
(I want to repeat, this was a single user exclusively personal domain. Writing to a friend, to grandma, to a colleague)
I was probably in a bad ASN, but at that level trying to find a good block you’re just rolling dice. I wasn’t willing to play anymore.
that they exist in part to force the hand of small companies and users to simply submit to a big player for their email is something i have long considered.
DISCLOSURE: I proudly run my own email server.
I have the dumb idea of trying to make SMTP as cheap as http. Make spam expensive using proof of stake.
I find it frustrating I have to pay Amazon to send text for me. I was going to setup my own SMTP server but it seemed like too much work.
Maddy (https://maddy.email/)
Postal (https://docs.postalserver.io/)
Chasquid (https://blitiri.com.ar/p/chasquid/)
That's already a thing. Hashcash [1], the PoW algorithm underpinning Bitcoin, was originally conceived as a method to prevent email spam.
The real benefit of SPF is for outgoing mail. People no longer forge my domains. It stops backscatter. It has almost eliminated the mistaken spam reports to my ISP by people who don't understand mail headers.
How is this a "barrier to entry"?
contact@ sounds off, like I'm a corporation. email@ or mail@ I kind of like, but I'm afraid it sounds "confusing" (is that in my head?). application@ or job@ is not bad, but a bit specific and not one I could use all around.
But when I give the mail directly to a given company, I use companyname@mydomain.com. That lets me track how I'm contacted, and sometimes it starts interesting conversations.
I’ve long been interested in self hosting, but constantly tweaking and never got #3 stable. After iCloud made it easy to do #1, I pushed my parents into using it and gave them my domain (@lastname). Now I just share it too, since it’s just too easy.
I encourage everyone who wants to change to something they control, or uses a paid iCloud tier to set up their own vanity domain for iCloud. It’s so easy and lets you own the identity, which is a critical part and portable. Not trying to shill anything, but it took 10 minutes and is offered by apple, so every not tech-savy person has heard of and trusts them, so it’s easier to convince others.
If you put some decent effort into making sure that you don't send spam, try to monitor if anyone thinks you send spam and react when someone complains that you send spam (and stop it), it works.
In my experience people telling these stories often do send spam, but they don't believe they do. ("It's not spam, it's a Newsletter. No, it has no unsubscribe link. These are people that agreed to be put on the newsletter by clicking on some ToS they never read, and they can unsubscribe by some arcane mechanism that we will make as complicated as we can. But we're definitely not spammers.")
But what is a problem is providing a good enough web interface, search, and so on.
However, one of my servers IPs is on a Microsoft blacklist since many years now. It sends <10 messages / day. I've tried every unlist form I could find, even called MS but it does not get taken of that list and they "won't disclose why". I'm routing SMTP to MS via another relay now :)
How do you do this? Could you share details on the setup?
transport.db:
hotmail.com relay:[relay.server.tld]:587 # and other domains
main.cf:
transport_maps = hash:/etc/postfix/transport smtp_sasl_password_maps = hash:/etc/postfix/relay_passwd
relay_passwd.db: # if necessary / not authenticated by IP relay.server.tld user:pass
The relay can/should rewrite the Return-Path to pass SPF. It's no problem for DMARC as the DKIM signature added by the initial server still authenticates it.
It requires manually adding domains of custom 365 installations to the list - at this size I do this manually, but should probably be automated "on bounce" or maybe even by a smart rule based on the MX record.
In Exim4 it's also possible to conditionally rewrite based on for example the recipient domain.
Well, I put in more than some decent effort, and I didn't get it "to work". I detailed my efforts here:
https://www.attejuvonen.fi/dont-send-email-from-your-own-ser...
Please stop spreading falsehoods. If you were able to somehow get your own email server to deliver email to Gmail and Outlook, great, good for you - but stop pretending that anybody can do it.
Yes, that's probably true. I've been running my own server for 20 years now, and I guess that in itself helps with getting my mail delivered (apart from t-online, but who cares about them). At some time I also hosted some mailing lists, but I quickly abandoned that because that's a surefire way to get your IP blacklisted sooner or later. If you set up a completely new mail server, there probably is a lot of luck involved, and I wouldn't recommend it to anyone, at least not for your critical business mails. I pretty much keep doing it only out of nostalgia, it doesn't really make any sense otherwise.
A few years ago we had problems, but then I realized some of the emails sent from our servers had non-ascii characters in headers (subject, from, to) which caused email-providers to distrust our server. Using encoded-words syntax ("=?UTF-8?B?" + BASE64(text) + "?=") fixed that problem:
Most of the time, delivery problems were of my own creation. Like running out of disk space or accidentally disabling TLS.
Once in a while, Microsoft would start swallowing emails or Google would push everyone to use DMARC.
But overall, the experience has been very pleasant. I host my mails, I own my data. I am not shy of using Google, but my work is not defined by their whims. When Google tells me I ran out of space in my account I just delete stuff because I have copies of everything outside of Google infrastructure.
It requires some effort to maintain and understand, and I’ve had a few deliverability issues over the years but they are generally with niche providers. I’ve never had trouble sending mail to the big providers.
Every time I read comments about the impracticality of self-hosted email, I scratch my head. Maybe I’ve just been lucky.
>Every time I read comments about the impracticality of self-hosted email, I scratch my head. Maybe I’ve just been lucky.
I feel the same. I've had one or two hiccups but smooth sailing for the most part. I'm also happy to provide receipts that show how the recipient's mail server is responding when I send the emails. It's a powerful tool to say, "your mail provider is misbehaving, look!" They will wonder how many people tried to send them email that didn't get to them.
What ISP should someone choose?
I'm doubtful a default block would work, as that would even penalise the 'big boys' of email when they make basic network changes and piss off existing customers of both sender and receiver... Its easier and logical to conclude something without reputation yet is therefore sending too few mails to be useful to a spammer.
I've had good experiences with smaller ISPs (currently Mythic Beasts). In contrast, OVH was a poor experience.
I find that reputation (beyond the known "block-lists") appears more likely being tracked for the whole AS number, therefore a lot more to do with your "neighbours" than anything else.
What matters most is if the IP address they issue you has been blacklisted for spamming. DigitalOcean is fine but you need to check the IP address before you do the work of building a mail server. Some of their IPs are on a lot of blacklists.
If it's only on a very few you need to look into who's blacklisting it. There are some that offer a way to get delisted and make it easy, there are others that block pretty much every IP address DigitalOcean has (or large ranges of them) and they won't de-list anything within them. Many of those blacklists are managed overseas and not used much in the U.S.
No matter the ISP you should check the IP address they issue for a VPS before you build the email server.
My personal gmail account is full of spam and emails I do want from email lists end up the spam folder randomly.
That being said, now I monitor and auto-ban failed authentication attempts to smtp/imap (among others) and running the service is fairly low maintenance.
But the morale of the story is that you are only one weak password from one of your users away from your mail server getting blacklisted as a spam server. So while I think it is fairly easy to run a personal server, running one for a small organisation is another matter.
That's about as easy as it gets but it still requires some work and you need to check the IP address DigitalOcean issues to see if it's blacklisted before you set it up.
Google makes it easy to get whitelisted. Microsoft email services (Hotmail/Outlook) are a pain though. I tried to get through their process but got nowhere. Other services I had to submit a request to get de-listed. So it does take awhile to go through all that.
Still, I prefer that to hitching that wagon to a 3rd party provider like Google, or any other.
Before I set mine up the 1st time I'd been screwed a few times by 3rd party providers. The last one, I can't recall which, but it was either "MailChimp" or whomever bought them, that I'd configured an app to use and almost as soon as I'd released it they announce they'd been acquired and I would have to use the new services APIs, and of course they cost more, and their services were geared towards mass mailing, and that's not what my apps do, and their API sucked for my needs.
It was about 12 years of dealing with 3rd party bullshit that motivated me to set up my own email server.
If you just want to fiddle around with one to get a feel for it Mail-in-a-Box is a good place to get started: https://mailinabox.email/
I've been running my own mail server since mid '00s. Initially hosted with one of West coast Canadian colos and subsequently moved to an EU colo. Had some deliverability issues with Outlook and Yahoo, but these were episodic and rare even though I set up DKIM only last year and have been running with just SPF and DNS/PTR before that.
I know at least a dozen of others with similar setups and timelines. But we all use dedicated colo'ed boxes on IPs from clean netblocks that weren't previously used for shared hosting. I strongly suspect that attempting to run a mailserver on Digital Ocean, OVH, 1and1 and similar mass-hosting providers will not go well. Just like it will be an uphill battle to run it on a residental IP.
What did I generalize, exactly? Parent poster was claiming that anybody can set up a mail server with good deliverability - that's a generalization. I said good for them (acknowledging they managed to make it work) and said that I also tried and couldn't make it work - therefore, clearly not everybody can make it work. Did I not argue against generalization there?
I run my mail server on Linode, no issues at all.
I do it as well and apparently so do many others.
Not sure why it seems problematic for some, but it hasn't been an issue at all for me.
I've run a hobby website for about 15 years that does not even have a newsletter of any kind, and includes "stop sending me emails" in each transactional email (all users are double opt-in verified), and password resets are still not delivered half the time to gmail addresses.
Frankly, I'm shocked you've never been arbitrarily blocked and I find your insinuations offensive.
The last time I was getting blocked it was the solarwinds fiasco where their internal mail tunneling/forwarding and filtering setup broke all DKIM and suddenly solarwinds users like NOAA.gov were rejecting me and adding me to naughty lists. There was no fallout for the megacorps and their broken setups. There was only damage to independent mailserver operators doing the right thing.
I've run my own email for decades and I've designed and run some pretty big commercial installations.
As a small provider, you run the risk of existing in a netblock used by other people sending spam. A small co-op I ran encountered this problem once. They were operating on the cheap and while they weren't sending spam their neighbors had been.
Even as a large provider at a billion dollar company, figuring out delivery issues is a huge pain and generally not worth it. There are unofficial professional postmaster meetups around the bay and these can be helpful in getting escalation contacts to fix issues, but even with entire teams of people dedicated it's a lot to handle and is usually worthwhile to outsource the work to other companies who already have these types of relationships established.
Unlike the person to you're replying to we had no issues with Google or Microsoft (once we did the requisite things) - it was Yahoo (and the people they provide email for) and then multiple mid-size organisations who used IP based block lists. At one point our mails were being rejected by our local NHS trust, the London Fire Brigade and a mental health agency we make referrals to. None of this was complicated to resolve but it was energy that could have been better used elsewhere.
I'm not usually part of the "let's go cloud without doing any cost-benefit analysis" movement but with email delivery I was happy when we could go back to routing via Office365 again. If a recipient decides to ban Microsoft's IPs that's usually going to be a bigger problem for them than me.
If you can have a high degree of confidence that no outgoing smtp spam traffic has ever been emitted from any of the other IPs adjacent to where you're hosted, the opaque blacklists of the big mail receiving providers (gmail, etc) are much less likely to consider your legit traffic as spam.
I've been running mailservers using free software for 20 years. I've run two for personal use, and several for groups like companies. In the old days, you could indeed throw up a server, and provided you don't spam, and you're not in a bad neighbourhood, outgoing mail would be accepted.
In more recent years, my experience has been that it takes time for a new mail sender to be acccepted; could be a year or two to build reputation. That's assuming you do everything right.
My personal mail, by the way, has been on the same domain since about 2001. I've quit running a mailserver now. My small ISP runs a setup that's basically what I would have built, so I use that; the support is excellent. But it's still on the same domain.
Last company I was at ran their mail on their ISPs mailserver. The ISP got taken over; service deteriorated, to the point it became unacceptable. So I built $EMPLOYER a mailserver; it took me longer than I predicted, because the bosses had all kinds of finicky requirements (don't they always) that I had to figure out how to provide after the fact. But that "artisanal" server beat the bejabers out of the ISP system; it was fast, reliable, and when anything went wrong I could fix it - which that ISP couldn't.
Also running my own servers for personal and business, and working well.
But when we tried to use one of the large VM providers the experience was much less reliable. Despite ensuring the IP was not on the various block lists etc. mails would be accepted and silently discarded by recipients ISPs, perhaps due to the level of abuse of these IP ranges.
Yes. I do get that impression from most complainers.
I send from my own domains, and if I sent it, I wrote and addressed it personally.
If I send an email to a corporation’s customer support, or to a distant relation, or to an open source mailing list, and I don’t get a reply, it could be a delivery problem - or it could just be that they didn’t decide to reply.
At work I ran email servers professionally and with good deliverability for years. My own email server was arguably longer lived than those at work, just much lower volumes. IP block was clean, DKIM, SPF, rDNS, etc. all setup correctly.
I thought I had no deliverability issues. I interacted with mailing lists regularly, the odd email to friends and family and I was firmly in your camp until I had to deal with a death in the family.
I think this was shortly after Microsoft BPOS became Office365. It became very very clear very very rapidly that to certain orgs I just wasn't hitting the inbox. And there was jack shit I could do about it. That was the end of my mail server, and it's certainly got worse over time.
The unspoken "you silly prick" gets louder every time this happens.
The big players all have a process and followup within days.
There's a tendency to perma-reject e-mail coming from "not seen before" domains despite the e-mail passing FCrDNS + SPF + DKIM + DMARC validation, which makes it difficult for private e-mail server users to get through to people.
I’d argue that the vast bulk of email is sent from dedicated providers like Sendgrid which are built on the same tech that might be found in any given ‘artisanal’ on-prem service.
This is a popular opinion on HN but it doesn't seem at all inline with reality. Email isn't exactly a real money maker for anyone. And the amount of email spam and abuse is immense. Filtering out most unknown providers is unfortunately extremely effective. Almost all spam wiped out with a simple check.
Maybe the ideal solution would be to let you link your custom email domain with a google account so you can have your google account vouch for the legitimacy of your custom domain. But even then, some of the time your email server actually is just blasting out spam without you knowing it.
Literally every month I got blocked again because my server did not have enough reputation. Kept logging tickets to get it unblocked and then a month later it was back. One time I did manage to get a personal email back from a guy in India. Said that it was because my mailserver did not send enough legitimate mail for their algorithm to trust it.
So the lack of spam is not enough anymore to be blocked. You actually have to send a load of legit traffic to build up 'reputation'. Now just being a small time sender is a problem. This way the big players can just carve out a bigger market for themselves. They basically break the decentralised concept of email by doing this.
In the end I moved to O365, which felt bad because I didn't want to reward them for their behaviour. But we moved to it at work too and I wanted an instance with full admin rights to explore. My contract is up next year so I may change then if I can find a party that does it well and ideally cheaper.
In other words, a small self-hosted email server will be considered a spammer until it starts sending out large amounts of email? Maybe that can be automated...
I have not had deliverability issues for years with my Kimsufi (OVH France) server. While I am confident my server is well configured using best practices, I suspect some of it is also just luck not to be in the same IP block as a spammer.
I heard Kimsufi is indeed pretty bad as it's so cheap people tend to use it for 'throwaway' purposes. It's basically the white label budget brand of OVH :)
Maybe a hosting coop could be an option? Large enough for reputation but ethical enough to still federate with smaller hosts?
You can actually get a free test tenant from MS for 3 months but setting up a real production environment is much better than doing some tests.
But yeah I feel lousy about it.
Or it’s because there is a near infinite number of domains so it’s relatively simple for spammers to avoid bad rep blocks by grabbing new domains and starting fresh.
Speaking of which, how long before it won't be possible to host your own web server?
On another note: the biggest source of spam is gmail itself, and guess what, that makes it to my inbox just fine, because what could possibly be wrong with someone using google as their source. Spam was annoying but it was never an actual problem. The consolidation of the internet into a handful of players is a problem.
I don't get this one. How do large email providers make it difficult to host your own email?
I host my own email. It was a pain to setup so I try not to touch it since it is running fine. Setting up email on your own server is just complicated unless you install server management software. I am not sure big email providers are to blame for this.
By randomly marking your email as spam without any recourse. This may be because they blacklist your provider en bloc, your IP address or some subnet, because they feel like it, it's Tuesday or because their spam filters suck.
But it happens and it happens often enough that running a business in that way will cost you money, sometimes lots of it.
So you're saying that anything can get you blacklisted if you're unlucky enough? I think that's the point of the people you're arguing with.
At this point we just need to figure exactly how unlucky.
Correct.
I'd like to describe how badly this is implemented:
I run my own mail server and I have a 15+ year history of emailing (mywife)@gmail.com.
On a regular basis (mywife)@gmail.com will email me, and I will respond to her email and my response will go to her junk/spam folder.
And there is no alert, no bounce, no notification.
Let's unpack this:
Google (gmail) knows that these two email addresses converse back and forth, regularly, with a 15+ year history. Google knows that their own user initiated this conversation. Google knows my email is a response to their users email. Google knows my address has never been marked as spam/junk.
So, what kind of unimaginably bad heuristics would have to be in employ to allow this to happen ?
To be honest, this wouldn't bother me that much - I don't think google owes me anything and my wife doesn't pay for their service. What makes me so, so angry is that they behave this way without any notification or bounce email.
That's just shitty.
This is every Google product in a nutshell for me. Their "algorithms" are absurdly bad in every category.
Also one thing - if people actually want your email they will contact you if they don't get an expected email. If they don't want your messages it is spam.
By not delivering mail sent by your mailserver to mailboxes hosted by them. There's not much use for an own server, if your mail won't be received by most users on gmail or hotmail.
[citation needed]; is this actually going out from gmail or does it just use gmail return addresses?
I too used to run my own email from about 2000-2010, but the maintenance overhead is quite stressful especially because it always happens for critical times or critical emails.
You are of course welcome to not believe me.
شهر مجاني عند الاشتراك السنوي $ الأسئلة الشائعة
In 1979, LA residents were wearing masks — because of smog Los Angeles Times staff photographer Boris Yaro photographed Sera Segal-Alsberg on Crescent Heights Boulevard in West Hollywood Segal-Alsberg, an artist-instructor, was en route to teach a class at the Los Angeles County Museum of Art
للمزيد من الأسئلة
— In another sign of live entertainment’s rebirth, Bruce Springsteen returned to Broadway over the weekend
يقوم الموظفين بتسجيل حضورهم ، انصرافهم الشركات العصرية مع الاستفادة القصوى من الإمكانيات الهائلة التي تقدمها لنا تكنولوجيا العصر أو الهاتف المتحرك ( الجوال ) أو إذا كنت تستخدم الحاسب فيمكنك استخدام أو يمكن للإداري تحميل هذا التطبيق على جهاز تابلت اشترك في النظام 10
Diverse yet divided cities
واحد أو عدة أجهزة ثم يضعها اشترك في النظام تسجيل دخول فقط في الأفرع المسموح له بالتبصيم فيها | أسبوعين كما أن الموظف لديك يستطيع التبصيم في ثوان قليلة بريد إلكتروني هو نظام إلكتروني قوي وحديث يستخدم لتسجيل فهل يمكن استخدام النظام في جميع هذه الأفرع أدخل بيانات الأفرع إن وجدت الموظفين
Experts say the Delta variant poses a greater chance of infection for unvaccinated people if they are exposed The variant, first identified in India, may be twice as transmissible as the conventional coronavirus strains It has been responsible for the rise in cases recently in India, the United Kingdom and elsewhere
في مداخل الشركة أو أفرعها المختلفة يمكنك الاعتماد على أي جهاز إلكتورني حديث أو حتى قديم في تسجيل ومتابعة تبصيمات الموظفين ، إعرف المزيد اشترك في النظام مجانا , أدخل بيانات الشركة والموظفينAnd yes, it's genuinely from Gmail; valid SPF, valid DKIM, came from a Google IP address, etc...
To say the biggest source is Gmail might be technically wrong though - I suspect there's a large volume of spam that Migadu (my provider) is dropping before it even reaches my inbox, i.e. emails that it is 100% sure are spam and it can just drop. Nevertheless, an overwhelming amount of spam I observe/have to deal with is coming from Gmail. Second to that is outlook/hotmail.
This. It's more likely to be survivorship bias -- the gmail emails happen to survive because gmail is more trusted.
> Gmail obvious spam still #1 in the quarantine folders..
-- Michael Peddemors, President/CEO LinuxMagic Inc.
https://www.mail-archive.com/mailop@mailop.org/msg14526.html
It's increasingly getting harder and harder. Recently I was trying to watch a TV show with my friends using a self-hosted Plex server, which was located in one of my friend's house, connected via a gigabit, albeit residential link. Another friend was using LTE internet at that time. He couldn't watch the show, because his connection was so slow, but when he did a speed test the download speed was good enough (100+ Mbit).
Turns out the mobile carrier was throttling connections to select IP ranges to about 1 Mbit (we tested that with a few other IPs). I reckon it was to cripple peer-to-peer protocols. So I guess it's a matter of time until you will be allowed only to connect to certain IP addresses owned by the biggest companies (AWS, Azure, GCP) and nothing else.
Maybe its just a matter of time for some. For me personally, I could not possibly care less if all the free mail providers blocked me some day. If something is important I can call people and tell them to go to https://mydomain.tld/theirName/ to grab files. I have used this method with non technical people including lawyers without issue. They prefer of course to use their own secure portals. I do acknowledge that running my own mail server may get more expensive with time as I may have to use providers that and more vigilant about keeping abusers off their network.
As for web servers why would I not be able to run my own servers? I can rent VM's, physical servers, racks, cages.
I am just speaking for myself but I will never give in to the bully anti-competitive behavior of the likes of Google and as for ISP's I will not use one that blocks ports or protocols. If there is any blocking to be done it must be done by me. I would never fund an ISP that uses CG-NAT or rate limits something by protocol or port. I realize some people have limited options but at least in terms of blocking and rate limiting, those ISP's are shooting themselves in the feet given that providers like Starlink and various 5G providers will be more common place soon.
It's not as feature-rich as Gmail, and webmail with your own encryption key is not usable, but desktop (Thunderbird) and mobile (K9 mail) clients fully cover my use cases. Cheaper than Google Workspace, too.
Walk through a computer interface with a 90 year old sometime. It is eye opening. Both webmail systems were utter design hell.
The list of stuff that tripped my friend up is long. Two examples: Gmail has pencils everywhere and at least two different styles to compose a message (chat style, big screen compose, reply style too I think.) Microsoft’s product has a typeahead for the To: field that ignores your contacts list and instead uses the institutional one, so typing “Anne” pulls up every Anne you’ve never heard of @youruni.com and not your friend Anne @gmail.com.
Gmail is also punitively fussy about receiving IPv6 mail but only on of its mail exchangers, so one in N mails get rejected. Great. Microsoft outlook requires you to scroll down each thread when opening it to see if “new message” meant the one at the top, or “new messages” plural further down.
You and I have become inured to this crap because we are comfortable solving problems with computers. For others, these products are very hard to use.
The one consistently brilliant client I use is the iOS mail app, via imap, to my personal mail host.
They're probably going to wonder why they are wasting their time with me when I can't even get something "as simple as email" right.
For personal email I'd probably consider it.
It's their system that is misclassifying their email, not you being on a "hipster" email system, whatever that means. At no point is it your system's fault so don't even begin to start phrasing it as such.
From your perspective, your email was delivered successfully.
But to the recipient, you claim to have sent an email that is nowhere to be found in their inbox or spam folder, and if they contacted IT (keep in mind that 90% of people won't even bother doing this much), they'd say it was never received - why should they believe you?
AFAIK there is some interface in business gsuite, available only to domain admin and not regular users, where a limited subset of the above can be done.
The same applies for inbound email too -- they claim they are trying to send me an email, or getting an email from something like a network printer fails. Good luck trying to find inbound email logs for the entire gmail.
Besides, there's a bounce email that describes errors.
They can't prove a negative, but they valiantly try, often using the very points that are excellent reasons to self-host as reasons we should all just give up.
"It doesn't work for me, so it won't work for you" is rather weak when the reasons aren't articulated, when generalizations are made without detail, when hardly any attempt at all was made to find the root of the problem. Your failure is not the same as my quarter century of success.
They also tries to make everything all or nothing, which plainly lacks any imagination. I wouldn't want an admin on my staff who tries something, then gives up at the littlest of hurdles.
There are plenty of issues, but there are also plenty of solutions. You're having issues sending because your netblock sucks and you don't have money for something better? Pay a few dollars to smarthost, for instance.
With Google and Outlook, you get no determinism, no accessible logging, no clear view about their filtering rules, no assurance that your outgoing email won't get blocked, anyway, because of the tremendous amounts of spam these monopolies allow...
We should encourage MORE people to self-host email, not less. We should never just assume the monopolies are the best we can do.
Maybe I missed it but I scrolled through the comments and I don't see anyone in this thread claiming the above. In fact, I see the people who had problems with personal mail server deliverability acknowledge that others may have success and "that's good for them."
I do and outlook.com still gives a shit and doesn't accept incoming emails. There is also no way to get their attention for this issue, as they simply don't reply to complaints about that. How to solve issues like that?
> We should encourage MORE people to self-host email, not less. We should never just assume the monopolies are the best we can do.
That's something I can wholeheartedly agree with.
>There is also no way to get their attention for this issue, as they simply don't reply to complaints about that. How to solve issues like that?
At one point, after getting one-too-many rejections for a particular recipient, I started sending the postmaster of the recipient’s service an email every time I needed to contact the recipient. That resolved the issue pretty quickly. You can always try annoying the postmaster.
Just personal emails. Volume would be somewhere around one email per month or so.
> Are you getting anything on your abuse@ address?
Nope.
> Are you getting any feedback on JMRP [0]?
That seems to require a Microsoft account and that's something I neither have nor want. I believe sending emails has to work without having to register an account for each provider you're interacting with.
> If all else fails, the mailop mailing list [1] can get you closer to someone on one of these networks to help.
Thanks for the tip, although I'm not sure if I'm the right audience for this list, as I just run a personal mail server with very low volume.
Funnily enough the "Best practices" section of the Mailop website contains a dedicated point (https://www.mailop.org/best-practices/#large-providers-gmail...) stating that there might be unresolvable issues when sending to large providers:
> If you want to send mail to recipients who have accounts at big email providers, be aware that all of the above cannot guarantee that these providers won’t reject your mail, put it straight into recipient’s spam folder or just silently discard it - they just impose their own rules on anyone and you virtually can’t do anything about it.
My guess is, that the reason for the problems is the same as quoted in another comment (https://news.ycombinator.com/item?id=29673347), stating that the mail volume is just too low for outlook.com to establish enough trust in the sending mail server.
If you look around on the internet, there are plenty of other people with the same issues with outlook.com. To me it seems Microsoft is doing something fundamentally hostile to small mail servers there. Interestingly enough sending to Office 365 hosted email addresses works just fine.
Yes it does require signing up with them. I see that you’re taking a moral stance on this so I guess the best action is to just ask people to not use Microsoft email products, which is perfectly reasonable in my opinion.
For what it’s worth, I do have an account with them and I am very small scale and don’t have any deliverability issues with Microsoft.
They ignore any argument you make or proof you show them that their servers are accepting emails and then silently dropping them. So that's basically the experience you mention.
However they will immediately unban your IP when you mention that you will recommend your customers switch away from Outlook email since it is unreliable.
That's for a totally clean IP with no spam issues.
(Of course I'm being facetious; other communication channels exist. But the idea made me think of the interrogation scene in The Matrix with Microsoft in the role of Agent Smith.)
Of course with Gmail I disable automatic image loading to avoid being tracked, but I want to take a more active stance against the practice.
I mean this as an honest question. I don’t really know variety of tracking pixels out there. Are there other similar things one should block?
Do you mean pay a few dollars to use someone else's email infrastructure? Or do you mean pay a few dollars more to set up a dedicated mail host in a better neighborhood?
> Entirely "on premise" email is now an inferior thing for almost everyone.
I disagree on this one. Placing your email with a big player means that by definition, they have access to your mailbox (with sensitive stuff hopefully encrypted). To allow that you have to trust the big player and the countries where they reside.
They can drop you any time for political reasons, for dealing with a country that is considered an enemy of the host country of your provider. They may sell out your data.
You may still choose a big player, but understand how screwed you are.
"Your account has been suspended for suspicious activity."
The author of the post should have included that in his post.
Sort of, but not entirely true ...
If you run your own mailserver then users of that mailserver can send and receive mail, to each other, without traversing a network. The mail never goes out on the net. That can be valuable/interesting.
This is true in both the webmail use-case (the text goes to the browser, ephemerally, encrypted with SSL) or the terminal/console (alpine) use-case (the text goes to the terminal, ephemerally, encrypted with SSH).
There's a certain cloud storage provider I know of whose internal / intra-company emails have never traversed the Internet ...
If yes, then I sense business potential. I for one would be willing to pay a lot of money to access my inbox.
One reason that I continue to run my own email server after more than a decade of trouble-free operation (thanks Postfix and Dovecot) is that it performs better than my Gmail account, which I maintain as a 3rd-level backup and for some email lists. Delivery and receipt from my personal email is so fast that I can use it for real-time conversations with anyone else on a good-performing server, almost like chatting on WhatApp. This doesn’t work with Gmail, because it takes so long for a message to leave their servers. Plus on my own system I don’t get the spam false positives that plague Gmail.
Those are only two ways that a personal server outperforms Gmail. I can do plenty of other things with it, because I control it, that are impossible with Gmail.
Agreed, but the same observation could be made about most blog posts. The author is expressing his opinions based on experience and judgment. This particular author has a good reputation so HN readers take his opinions seriously.
The reason I and many others maintain our own email servers is not to be “cute” (a word used in the article) but because we want superior usability, features, and performance over what the major providers can offer. Otherwise, why would I bother? The only issue is that some people claim to experience delivery problems. Others, such as myself, say that delivery is at least as good as from major providers.
And receipt is far better than when using a big provider. Unlike the unfortunate users of hotmail, I actually get all my email.
EDIT: And not only does (for example) Gmail run a poorly performing email service, but their web client for interacting with email is broken: https://lwn.net/Articles/837960/
I configured postfix with:
recipient_delimiter = .
I know that you can do the same thing with google addresses using + as a delimited, but the + sign is often not allowed in dumb email checks. Also spammers probably know about + and strip it automatically anyway...
foo+anything => redirected to foo
foo.something => redirected to foosomething (so . is not the same as +)
unless they use BCC
I eventually ended up at Fastmail, as they let you build custom Sieve scripts that can do this kind of remapping without having to run your own mail server.
Running your own email server and domain, for people who like systems, is fun. Just like people who do their own oil changes and car maintenance, or people who build their own furniture, etc.
Interestingly enough, this suggests there are some startup opportunities for folks who want to make this stuff a bit easier. Three things I think would be interesting side projects would be 'spam killing' (Barracuda does this as a service for Enterprise, I bet you could do it in clever ways for individuals), "post office" which is a known good relay server with mail agents that you can forward your mail through (think Lets Encrypt but for mail delivery), and a remote access client for phones. Alternatively an AWS offering of a packaged mail server (think WPEngine but for mail) has possibilities as well.
(also worth mentioning: email protocols were explicitly conceived so that uptime is not a worry)
That is both true and misleading. Once the datacenter and all surrounding infrastructure (optic fibers, fuel pit, dedicated electricity lines, cooling equipment) and all server/networking hardware has been built, then you start having a better efficiency. If the whole cycle is taken into account, there's no way VPS can be as "green" as selfhosting.
A computer will usually take more energy to build than it will consume over its entire lifetime, so repurposing an existing machine is a good way to go (if you consider minerals-related pollution, even more so).
Also, when you're in a datacenter, servers will be changed every few years. For something as simple/lightweight as email, a 20y old computer will do just fine. A datacenter will renew its entire hardware a few times in that timeframe.
> A computer that's 99% idle is wasting most of the energy it consumes
That is true whether it's in a datacenter or at home. But of course you can share/mutualize resources with other people in order to mitigate this.
I think OP might have meant "receiving on port 25 is getting difficult" rather than sending. The spec requires servers to support unencrypted deliveries over port 25, even though almost all servers use TLS these days.
Another one is to mention hosting coops (libreho.st/chatons.org) and how they could be employed in limited-network situations. On the web, we have SNI/eSNI-aware proxying which enables multiple servers to share a single IP without revealing their private keys to the reverse proxy. I don't know of an equivalent in the email world (because it's assumed there is only one MX with a canonical domain/DKIM per IP), but i'm all ears if you have suggestions!
Of course, we could mention onionMX and other key-routing systems (CJDNS..) but the problem is you need it to be supported on the other side as well, which is highly unlikely.
The worst part is, that you can’t use any fancy email clients, because they all use proprietary protocols and once again cloud services. But imap and activesync works well enough too.
The most important thing is, to get a clean IP address. Don’t ever try to host your Mailserver on digital ocean for example, their IPs have such a bad reputation that some providers even block them on network level. Their whole subnets can’t connect to them, no possibility to get unblocked at all.
Once you found a legitimate hoster, check that your IP is not bkacklisted at any major provider. If it is, try to get other IPs, until you get a clean one. Don’t try to go through unblock-processes, that often won’t work.
And then you need to set up your server well. SPF, DKIM, DMARC, …
What fancy email clients did you have in mind that don't support IMAP?
https://www.imore.com/best-email-apps-your-iphone-and-ipad
They do support imap, but they process your messages on their proprietary cloud service. What sane person would agree to that?
In the US there seems to be not a lot of concern about data protection yet, so there seems to be a market for those services.
DMARC does not provide signatures, DKIM does.
DMARC adds the DKIM 'alignment' requirement. Meaning that not just any DKIM signature will do, the public key (the DKIM DNS record) must be published under the administrative domain (the part after the '@' in the sender address).
DMARC also mandates SPF alignment (not that your should rely on SPF), meaning that the rfc5321.MailFrom and rfc5322.From address should be from the same administrative domain for the SPF to pass DMARC.
When either SPF or DKIM is aligned, you have a DMARC pass. Because SPF breaks with forwarding services, you shouldn't rely on it. DKIM + DMARC is the way to go.
Also funny that the author calls DMARC 'modern practice', since DMARC was introduced in March 2015, almost 7 years ago.
Oh come on, in terms of protocols, that's modern.
Your other points have merit but that's just a pointless dig at the author.
On the whole, just use a service makes sense in that the time you spend on email probably could be spent on things that make money. If you have an app that sends a lot of emails, understanding how it all works can be a very useful skill, too. So is learning to work with admins at big service providers.
The wild west days of email, with bang paths and "store and pray" delivery systems, those were fun. By the time `sendmail.cf` hacking was no longer a necessary skill, email had become industrialized. Today, why would you even want to try routing internet email through an RBBS net to WWIV net to some hackers custom Amiga board?
SMTP was the Ford Model T of electronic messaging. It slaughtered the previous visions of what the field needed to be. We can look back fondly at the older ideas and even re-implement their insights now, but the lessons of the market are written in big bold letters now.
I never linked it to WWIV though I was my area code coordinator for WWIVLink. That had to be around 1988 or 1989?
It was wilder before that, think "B news" times.
IMHO, gmail is the best company out there that can accept your emails. The most common issue with gmail is email land in spam but they learn quickly and very rare just outrage reject IP. as in, gmail won't relying solely on reputation of IP but based o sender domain or so.
Compare with hotmail or icloud or some random email hosting service, they will reject your IP just outright.
I would say it's definetely tough and stressful to run an email services for everyone, but if you run it for yourself only(so you know you're good and won't send out random spam), I will say it ins't that bad.
It's kind of a pain, because it's both not really a Google account, but also not a real Gsuite setup, and Google keeps quietly removing features from the legacy version of Gsuite. (You used to be able to use an external SMTP server to send as a different domain, but Google removed the UI to configure that in the free version)
I've been really hesitant to upgrade to modern Gsuite, because I'm worried about upgrading and having something go wrong in the process - Google support doesn't have the greatest reputation.
I've said this for over two decades: If you have a business, or even just a lot of important stuff going on in email (which is like everybody); it strikes me as insanity to not pay for the peace of mind that comes with "a human you can call up and say 'hey, why can't I get into my email' or 'hey, fix this please"
Versus what SO MANY PEOPLE use, which is "It's possible that your email will be removed from you entirely and you will have recourse because no one has a contract with you to fix it."
That stance would make sense if that were an option with google hosted email.
I ran mailinabox for a year or two, but eventually I just didn't want to maintain a piece of software I didn't understand where the documentation seemed actively hostile and presumptuous about me having read all the other parts. I'm sure the postfix docs make an okay reference, but understanding it as a whole, god no. I'd rather do Kubernetes from scratch.
Fastmail is just fine for me.
The docs for magic word projects never to seem to prioritise essentials. So [obscure feature someone last used in 1984] gets equal billing with [essential fundamentals] and you have no idea which is which because - you haven't understood the docs yet.
I'm still running my own servers. I sorted out the spam issues, and they're basically zero maintenance now. But it certainly took a while, and a fair amount of copying other people's ideas of what a config file should look like, with plenty of trial and error.
Maintainer of Mail-in-a-Box here. I'm sorry you had that experience. Definitely was not the intention of the project to be hostile (but I can see how it might come off that way).
For most of the time it's been smooth running, but I did have to do maintenance on the server every year or so, just in time to forget the intricacies and having to relearn them again.
Yeah, a few hours a year on that is not much. But there are many such small "auxiliary" things/chores and it adds.
There's so much things I would and could want do myself, and nowhere near enough time to do them. I have to pick my battles.
And figuring out how to fix sender rewrite to enable mail forwarding with SPF without accidentally allowing spam is not very high on my list of important things in life.
My biggest problem with mail delivery is sending mail to Microsoft properties. I've had to resort to sending those messages via SES.
Open source webmail solutions suck, so now I'm paying fastmail and forwarding incoming messages there.
AS3150 is NTT, a large backbone provider: https://bgp.he.net /AS3150
RamNode runs AS3842 and AS198203. We have contacted them before - they don't know why nor care why their system is raising issue with AS3150 in regard to emails from our network.
But this issue isn't exclusive to our network either, and no other major email provider blocks emails like this.
My only complaint was the graylisting but I quickly resolved that with a configuration file.
It would be nice of you could just install an email program that will set all the right settings for you. DNS, database, roles and rights, certificates, firewall and so on.
There is server management software that can do this but then you have the same problem: it is just complicated for most people.
First, SPAM filter is way overrated. I have next to zero SPAM filter, and am doing just fine. Yes, I got lots of SPAMs, but the volume of real SPAMs is dwarfed by the volume of ads that would pass through SPAM filters anyway, so why bother.
Second, yes, open source webmail is lacking, that's why I wrote mine: https://github.com/derek-zhou/liv
Lastly, the biggest pain I have is sending email to big providers such as gmail. I have everything setup correctly, DMARC, SPF, you name it. And my server is not on any block list that I can find, and yet they put my emails in the SPAM folder from time to time. In the name of fighting SPAM, they are sabotaging the original internet experience for everyone.
Not much has actually changed about the complexity of running a mail server in the last 20 years --- if anything it's gotten easier. What's changed is there are other, polished, turn-key options now. Great. (Those options tend to have spam policies that aren't friendly to the independent servers, but that's life.)
Choose to be artisanal.
(I'm the primary maintainer of https://mailinabox.email/.)
I ran my own mail server for 20+ years, finally giving up a couple years ago. I strongly disagree that it has gotten easier. As the article makes clear, it's a much more complicated world. Things that have happened in the last 20 years include SPF, DKIM, DMARC, and the rise of providers like GMail. And if you really care about owning your bits, in some ways colocating hardware has gotten harder now that VMs are hugely dominant.
The new requirements to be a good mail server are significant work to understand and implement. The feedback loops are also poor: it's hard to know whether you really have them right.
But the real killer for me was opaque major providers like Google. Occasionally, they decided they didn't like my little mail server. I and a number of other sysadmins couldn't find anything wrong with my setup. But mail wouldn't arrive. I even had SRE friends inside Google and they couldn't find out anything; apparently the GMail folks are very secretive.
There are only so many missed business opportunities and disrupted personal relationships I was willing to put up with for my personal taste for running my own servers. Eventually I hit that limit and switched everything over to Fastmail. For me personally, it was a great decision. It's cheaper and more reliable, and never again will I have to get up in the middle of the night to go to a colo. In contrast to my spending a few hours here and there, they have a whole full-time staff sweating deliverability. It's great!
If people think running a mail server is fun, I say go for it. But even there I'd strongly urge them to consider whether "this looks fun" is the right spirit to bring to anything important to their lives, and whether it will stay fun when it breaks at the least convenient time. So maybe keep it fun by using it only for things that don't really matter to you.
Right, of course. The protocols are more complex. (Add TLS, MTA-STS...) But whereas 20 years ago you _had_ to start from scratch and understand the whole stack, today that's just not necessary. There are numerous projects that make running a mail server readily possible without knowing e.g. the sendmail configuration macro language. And there are many many more good resources to learn it all if you want to know than there were 20 years ago. It is both a more complex technology and also undeniably easier for people to actually do it.
I wrote the email chapter for the book "Internet Secrets" in 2000, and I ran my own mail server 1997-2019, so I have a pretty good sense of what the landscape was like then versus now. QMail and Postfix were both out before 2001, so you didn't need to know sendmail at the time. You just had to be able to configure the mailer to get things up and running. And given that there were decent Linux distributions available, the technical challenge wasn't high.
The difference now is that from there, there's a lot more to understand if you actually want your email to get anywhere reliably. It's complicated, subtle, and much harder to resolve problems when you get it wrong. At the time, the biggest problem was bounces. Now deliverability has become a dark art.
Just out of curiosity, when did you start running your own mail server?
I've never used gmail itself (that model doesn't fit my mind), but O do use that Google account for some minor stuff. Unfortunazely, I've repeatedly gotten email targetted at someone else having same first name initial and same last name on gmail (address is in the form of FLastname@gmail.com). I've usually been able to get through to those people to stop them and to get them to reach their targets, but in the last 24 months, a lady from Michigan is repeatedly giving out my email address for everything (I've got covid appts, doctor appts, movie tickets, responses to home buying inquires...). I have no idea how to stop this: this would've never happened with any provider that's not owning like 60% of the market.
I am constantly annoyed and I've considered both stopping mail forwarding from this account to mine (but then I might miss that YT premium notification) and I've tried reaching out to many humans on the other side. But I've so far resisted the urge to cancel those movie tickets or vaccination appts, but things just keep coming in.
I can't imagine how are people not overwhelmed by wrongly targetted email: there's more of it than spam I get on my personal server, so spam filtering would definitely not move me towards gmail. And actual spam also gets through on gmail!
She may be getting it at f.lastname@gmail.com.
I signed up for Gmail the first or second day it came out with first.last@gmail.com, but hardly ever log(ged) in over the years. A little while ago I did go into it and noticed getting a bunch of message to firstlast@gmail.com.
Now Gmail allows for 'customized' addresses in that you can drop a period anywhere and it will still go to your address. But this raised the risk of one person signing up with first.last and another person signing up with firstlast. Supposedly this is prevented, but I think that they did not catch this situation in the early days of the service, and so a bunch of OG accounts have cross-contamination.
To even use that gmail address I need to basically whitelist senders and filter them into folders and ignore the inbox completely.
Gmail, like so many of Googles services these days, is an absolute mess. Features no one asked for, blatant spam that gets through their checks while your actual emails go to the spam folder, and a constantly degrading UI that seems to be an experiment in how much you can annoy the user.
That's a user problem, not a technical one. (there is, of course, an XKCD for that).
I've got a popular initials/surname combo and I have a number of doppelgängers giving it out. The one I feel most sorry for is the trumpist and his scary NRA/pro-gun mail. It's really fierce stuff, I'm glad it's going into my spam folder rather than in front of a real human!
He got many "weird" messages over the years from messages addressed to a religious minister, to a pro-gun individual and to NSFW account that he didn't sign up for.
Here are the downsides to outsourcing your email:
- Who will read your mail sitting on their server? - Who will give your mail sitting on their server to anyone waving some legal papers? - Who can kick you off their server without recourse, killing many years of investment into your email identity?
Here are the downsides to rolling your own email server:
- If an evil hacker invades your badly protected server, and uses it for gadzillions of spam, a nicer provider will turn off your outgoing email, a nastier one will null-route you no questions asked. That’s why you should start your career as an artisanal mailhoster on a VPS -- you can always rent another one. It gets nastier when a homelabber decides to do it from home. Hacker invades, Internet provider cancels your account for violation of TOS. You usually don’t have much choice when it comes to a new hardline provider, and it will take a while. - A big danger are the countless blacklists. It is very hard not to be on any. Some will blacklist whole net blocks if one IP in the block misbehaves. Some are plainly extortionist; they want money for removal. Frankly, they should be taken to court.
The challenges there are rather different. Spam is less of a problem: in B2B the correspondent is well-known and will be whitelisted quickly, while one can pay for extra spam filtering if needed. User experience isn't a biggie because frankly the biggest users aren't the ones paying. The real issue is TCO vs data independence - and in my limited experience I still see plenty of organization which still run Exchange (or equivalent) on their own domains.
My users do not need webmail and I do not offer it. Calendars are done in local clients and shared through invites.
I went with DC powered equipment, with battery backup on the input. My internet and email can stand at least a 24 hour power outage. Its also all solid state and no fans, no moving parts. Its been very reliable, though I do replace the batteries every couple of years. I'm probably over due for hardware updates at this point.
I do have a specific beef with all the consolidated email providers. If one of them determines your SMTP server to be spam (false-positive), ALL of their clients now reject your email with little recourse for the admin. Just had this happen with a solution that rhymes with 365. Even their clients were clueless as to how to resolve it.
The ignorant always blame the informed - and it keeps working cause the informed can actually address the issue.
i got set as spam by google for the first few months but nothing since then. sure, one day i managed to spam like 200 emails in quick succession and that put me into spam but a quick "please select as unspam" solved it. i've been running this for like a year and it has been a good experience. i recommend people try this out, it doesn't cost a tonne
https://gioorgi.com/2020/mail-server-on-docker/
I am running it and require very little mantenance.
The documentation is very well done and I was able to setup all the stuff needed in a short time.
Also paid hosting solution tend to be very pricely if you need more than 3-4 accounts.
My solution instead required only some setup effort, less than one day.
Microsoft seem to disagree with you there: https://support.microsoft.com/en-us/office/pop-imap-and-smtp...
The search and the interface are entirely the fault of MS as are the lack of more subtle features such as mass emails (which I have to use often while teaching online).
Mass emails via distribution lists are a thing - https://www.wisestamp.com/blog/managing-distribution-lists-i...
As is sharing links to OneDrive content with a password on it: https://support.microsoft.com/en-us/office/share-onedrive-fi... ("Set password: lets you set a password to access the file. When a user clicks the link, they will be prompted to enter a password before they can access the file. You'll need to provide this password separately to anyone you want to share the file with.")
(Poor quality sluggish Outlook client, dropping important emails into Spam, not being as configurable as a custom mailserver, hit-or-miss search results, those are all things I can agree with, I'm not just defending it).
They've already delayed it a few times but they keep pushing for this.
[1] https://jschumacher.info/2021/05/running-a-private-mail-serv...
not mentioned yet?
One of the very important things is choosing what ISP to host your self-hosted email at. And the spam blacklist (or opaque/impossible-to-know) likely blacklist status of your IP at things like office365, gmail, etc.
Assuming for a moment that you are a person who is perfectly capable of setting up your own postfix and dovecot server.
No matter how perfect your rDNS, SPF, DKIM, DMARC setup is, and how flawless your theoretical postfix or other smtp daemon configuration is... If it's not hosted in the right place, outbound mail deliverability is the main problem you'll run into.
For the persons who are not ready to host their own SMTP and mail storage, I'm going to second the other suggestions made in this same thread that say a good first step is to control the authoritative DNS for your own domain, so that you can choose where to point the MX records at, and make an educated/informed choice of third party mail service provider.
Currently, there are only a handful of large technology companies in control of most of the world's inbox. Google is the first that comes to mind. At the same time we've just had some of our most highly publicized hearings involving these tech companies (facebook and google) as well as our first hearing on cryptocurrency and the larger web3 infrastructure. There is a rising public awareness and therefore political will to regulate these technologies and companies. This would formally fold google and the like into the USG despite their long standing less formal arrangements with the intelligence agencies.
The one way to make this regulation and upcoming legislation moot? Decentralization by any other name. Self-hosting of email servers, or distributed computing and storage with web3. Perhaps its is only my latent paranoia, but I can't help but shake the feeling that the glut of the 'don't bother hosting your own email server' sentiment is, at least in part, artificially amplified in order for the coming formal regulations to have more of an impact.
It wouldn't surprise me at all if google themselves was helping to facilitate this in order to steer conversation towards stalling any potential competitors as a part of the new regulatory framework. It is for instance a lot easier to argue that outlawing self-hosting of email servers (or requiring a license to do so) makes a lot more sense if you can point to a 'general public sentiment' that hosting your own email server is 'too complicated' and 'less useful' and 'less secure' and therefore would only be done by antisocial actors such as 'criminal elements' and 'terrorists'.
Seeing as we are at the cusp of a new distributed infrastructure movement, all this feels like preemptive damage control to me.
Google deals in information control with e-mail hosting being one of tools. They are also very efficient at presenting themselves as being right. So folks and companies having trouble sending messages to GMail, moved to GMail over time since it seemed doing good job, in opposition to their sending offering which seemed at fault.
Google seems to be incapable of not fucking up anything that disrupts their ability to spy. I believe that as soon as they figure out a better way to creep on people's finances and purchases on a global scale, they'll move on to killing e-mail globally.
Microsoft, in contrast, seems to be incompetent when it comes to e-mail. From my perspective this was a true statement since they touched e-mail. 'member Outlook Express? Dealt with Office365? Same shit 25 years apart.
Hotmail is an occasional pain for the self-adminstered email server, everyone else works fine IME.
Especially the "dockerized" Mailcow is reasonably easy to setup.
You will still have to setup SPF, DKIM and the other DNS records. But Mailcow is a solid package!
I gave up. In retrospect I should have used another smtp gateway (perhaps from my provider), but back then the thought never occurred to me.
For a time it was nice, (basically) unlimited storage, unlimited aliasses, being able to send 200 mb (only to my wife admittedly) easily. I did really learn a lot, which it what I tell people now: Try and do it to learn, but it's really unpractical. I now pay for email.
At the time I was a heavy Mac user though so I ended up switching to mac.com email, and then Gmail when Apple EOL'ed Mac.com.
Today though there's probably an easy turnkey VM or docker image well-configured to work without too many false positive spam flags... I hope?
These days, it's probably easier than it was, back then (about twenty years ago).
It was a nightmare. I didn't do it for a living, so I was consumed by the task. It screwed up my other work, something fierce.
Hell, about two years ago I gave up on self-hosting. Unwanted email got to be the vast majority of what I was receiving and spam filter software ate more memory than my itty-bitty host had, meaning I would have had to get a bigger, specific mail host. (As it was, the IMAP server was the biggest process running on my host.)
We want a free, fair, and open internet.
I have a static ipv4 at home, and my biggest problems are the following:
- residential providers won't delegate a reverse dns zone or set a reverse ptr record for you -- or at least my ISP (Fastweb) will not do it for a residential contract
- spam lists (spamhaus etc) will blacklist residential ipv4 pools by default applying what effectively is a prejudice (and defamation).
Some considerations:
Major providers will effectively do as much as they can to prevent other organizations (let alone individuals) deliver their own mail.
Google is particularly shitty in this regard: it regularly delivers my mail to spam despite having both SPF, DKIM and DMARC in order. My gmail inbox however is full of SPAM because google decided that I really have to look at those promotional email.
Microsoft is surprisingly good instead: upon rejecting mail initally they're going to direct you to an automated procedure to de-list your domain from their spam services, and it works. Kudos to microsoft.
SPAM, surprisingly, isn't really a problem: if you have sufficient checks for incoming mail (does SPF for the domain allows this ip to send email? do DKIM signatures check out? are they using SSL/TLS for their connection? et similiar) you basically won't receive spam.
Most OS vendors (Red Hat / Debian / Ubuntu) deliver postfix with poor cryptographical default settings, meaning that the default settings will connect to plaintext SMTP to deliver outgoing email and will not setup submission (tls) by default, not even with a self-signed certificate. In the time of letsencrypt being available, this is a dumb choice.
Running low-volume mailserver is surprisingly light on resources. A raspberry pi is likely overpowered for the task. You can use pretty much anything and it's going to work, as long as it powers on and doesn't lose data.
Running a mailserver is also surprisingly versatile. The possibilities are pretty much endless. You want to alter outgoing emails? No problem, look at PCRE maps and postfix's header_checks (or similar). Want to have mail aliases? no problem. Want a catch-all address? easy. Want to hook a service into mail delivery chain? look at the milter protocols.
For example, Chrome & Firefox together will remove FTP protocol support because "stats suggests very few % people use it"
But I think you can set up an email server for your own tasks, like sending yourself some notifications. It's also nice to know how this works in theory.
These are the reasons I included some basic e-mail setup in my book (Deployment from Scratch), although also advising to depend on some reputable IP addresses.
tons of serious players don't trust email at all today
some, like my health insurance + bank, are dinosaurs who are all but licking the molten shockwave of a meteor. But still -- they have real reasons they email me that my 'secure message inbox' has a new message. Oscar uses 'secure email powered by virtru' to tell me they're not going to reply to my reply to their email, wtf, huge indictment of oscar, but also indictment of email.
but it's not just dinosaurs -- amazon, who has had a tough month but is generally savvy, doesn't trust email. they won't send me itemized receipts because they don't want google to read them.
I half believe we're about to see a renaissance in self hosting for individuals + businesses. this article is calling the bottom.
Gosh...when people forget how to be a admin...
>takes a lot of work to create
True
>and keep running.
Not true
You should make sure your server's IP address isn't blacklisted. If it is, you're going to have major delivery issues with some email server providers (ESP). Some blacklists you can check are listed here [0]-[3].
I think my main problem now is the UCE Protect [5] blacklist. I think some of the major ESPs use their harshest blacklist from UCE Protect, which is their Level 3 blacklist [6]. This blacklist will include your IP if your ISP meets a spam threshold for any of their other IP addresses. This makes running a mail server on cheap hosting providers like Digital Ocean or Linode very difficult.
My conclusion is I should switch to a more expensive ISP that isn't in danger of getting on the UCEPROTECTL3 list or find an email forwarding service for a next hop destination for outgoing mail.
You can read more about UCE Protect here [7]-[10].
[0]: mailtester: https://www.mail-tester.com/
[1]: mxtoolbox blacklists: https://mxtoolbox.com/blacklists.aspx
[2]: proofpoint blacklist: https://ipcheck.proofpoint.com
[4]: outlook blacklist: https://sendersupport.olc.protection.outlook.com/snds/index....
[5]: UCEPROTECT: http://www.uceprotect.net/en/index.php
[6]: UCEPROTECTL3 blacklist: http://www.uceprotect.net/en/index.php?m=3&s=5
[7]: UCEPROTECT Blacklist Scam https://community.spiceworks.com/topic/2170592-uceprotect-bl...
[8]: UCEPROTECT: When RBLs Go Bad https://blog.sucuri.net/2021/02/uceprotect-when-rbls-go-bad....
[9]: ASK HN thread https://news.ycombinator.com/item?id=26064722
[10]: SQLite3 IP blacklisted: https://sqlite.org/forum/forumpost/bb61881d7a?hist
[11]: Previous IP reputation issue https://news.ycombinator.com/item?id=25437841
I mean I get it but it’s not good for the internet for people to equate all email with gmail.
Email together with all the other preconfigured goodies make it a totally worthy investment,the yearly cost, for me.
Not a huge issue, and worth it for me.
A big reason it’s getting harder to self host is because so few people do.
If anything roundcube is a better webmail client than many of the mail-provider ones. And that's ignoring all the ads and tracking that these come embedded with, even if you pay for them.
I haven't used gmail-for-organizations but if it's anything like the normal gmail interface then ... I guess some power users will prefer it? But in my experience many people prefer roundcube, because it's simple and usable. Not that it's perfect or better in all the ways, it's just from my experience and the users I talk to, it is just as good and fills a need that gmail doesn't.
Same goes for spam-filtering. It's not that spamassassin/amavisd/rspamd/postscreen/RBLs/whatever is 100% perfect, it just get's you pretty far, and from my experience also gmail, as the main contender, has varying success on how close they achieve 100%.
And even security is not magic. A large mail provider doesn't have access to magically different security tooling than everyone else. They have a threat model that is slightly different and their scale allows them to do some things that not everyone can. But wrt to one's userbase it's perfectly possible to be "just as secure".
Running your own org-mailinfrastructure is certainly not "artisanal" - for some reason this comes off slighly dismissive in the article - it's just that, as anything, it's work that you have to want to invest in. A trade-off where it often does make sense to outsource. But then email is not so different from any other service you want to provide.
... but then I see that the article seems to partially be writing off of the experience of using the U of Toronto mail system, which seems to be using squirrelmail and procmail. I didn't even know squirrelmail was still developed - this impression kind supported by there being no news between 2013 and October 21, 2021 on the frontpage https://squirrelmail.org/ while procmail is unsupported since approximately forever (it feels like pre-9/11 but I am not sure).
If you compare that experience with outlook.com - then I can certainly see why one could come to the conclusion in this article.
EDIT: Heh: my guess of procmail being pre-9/11. Wikipedia says: "Final release 3.22 / September 10, 2001"
Actually various big FAANG companies have very privileged access to vulnerability disclosures.
However, the threat model can make small mailservers way more secure. Breaking into gmail is worth billions.
Breaking into your personal mailserver is not worth the time of any skilled attacker unless you have very valuable secrets.
Super easy to set up and nothing in the article will be a problem. Your system will not be "artisanal". 400 people wasting time commenting on this guys dumb article.
Nice to have: DKIM DMARC
Get all that in line and it should work fine here. I run quite a lot of email systems. I run them in the UK and not Canada or anywhere else so can't comment broadly.
I've been doing this for 25 years now, so I think I have a fair handle on how email works.
You can stuff your artisans up your arse. Email is email - do it right or fuck off.
[EDIT: I give up, I can't remember how to format a list on HN and I can't be arsed anymore - I created two lists above - they should be fairly obvious and each one has a heading followed by a colon]
I don't think you understand the complexity of having your emails marked as spams on newer domains.
I've also been running my domain email for 15 years or so and configurations are easy but I still rely on external delivery services just so I get less chance of being flagged as spam.
I've been doing email for 25 years (not 15), I have a few ideas about how it works.
Have you configured a new domain for emails and got them work fine against Gmail etc?
There's several advantages for me, I can easily backup Maildir with find -mtime +90, tar and purge.
Using mutt to read my own mail makes filtering off spam very easy.
Maybe gmail has some advantages, somehow the domain looks more "professional" than hotmail or outlook addresses do. Can't explain that though.
If you run your own mail, you have a domain, so running your own web site comes naturally too.
SHAMELESS
“If you want to impress him, use the word ‘artisanal’ a lot. It’s how fancy people say ‘good.’ Artisanal cheese, artisanal soap…”
“Wow, where did you learn all this?”
“Artisanal private school.”
A lot of people will say, "no they'll blame you for it not going through", but that's rare. Most people will be receptive to your insistence that you're trying to send them mail but their provider is in the wrong.
Now, if you mean blacklisting in the sense that the server has shadowbanned you and is sending back "221 OK", then, again, you have an affirmative defense: "hey, you're mail server said it accepted the mail, can you please check with your provider on what they did with it?"
In either case, this is actually not a good thing for the recipient's mail provider especially if they pay for that provider. "Why did you accept the email from the sender but not put it in my mailbox? Who else have you done that for? Why am I using you as my mail provider again?"
I have a circle of influence... about 100 people. All 100 of those people would switch mail providers if I asked them to. And I think a lot of people have a circle of influence around this size as well. So there actually is an amount of control over these bigger mail providers. They will be receptive to "I just told your customer to switch because you won't let me email them" especially if it's widespread. In exchange for that "power", I make sure my mail server is as clean as possible and quickly respond to any notices sent to my abuse@ address.
https://interoperability.news/2021/12/eu-parliament-upgrades...
What a BS, wake up dude.
> usability, features, and performance?
you really make me laugh, it's so bloated it's barely usable.
It is my profession after all.
- Reverse DNS
- SPF record
- DKIM record
- DMARC record
- limit outbound number of emails to catch possible spamscripts
This is one thing the authoritarians like Biden (Clipper Chip, Patriot Act) won't want to fix. There will be no law that companies with more than 100 employees must accept mail from individual servers (they would still have the correspondence anyway, but it would be a start). There will be no law that all mail must be encrypted.
I strongly disagree on the motivations given by the article (the service you provide will never be on par with Google/Microsoft). This is plain false. There's plenty of webmail clients that look as modern as a Google/Microsoft UI. I even find the Outlook UI to be more primitive and spartan than many open-source UIs.
If running a mail server in 2021 is painful, that's because cloud giants have built a small mafia that, using the excuse of spam protection, excludes anything that doesn't come from a handful of mail services, period.
I used to have my own @mydomain.com mail server back in the golden age of the Internet, with mydomain.com pointing to my sitting in my closet. At some point, around a decade ago, mails sent from there started being refused - Spamhaus really doesn't like SMTP servers with dynamic IPs.
Fair enough. Back then I migrated my home server to a Linode instance. It worked for a while, but then even those emails started being sent back by some providers.
Fair enough - I thought. In theory anybody could still rent a Linode instance, buy a domain name, set up their mail server, and spam the world.
So I moved @mydomain.com under my managed (paid) ProtonMail account, installed a ProtonMail bridge on my server, and forwarded emails this way.
This should be alright, isn't it? I'm paying for a Linode instance connected to my name and surname. I'm paying for a domain name connected to my name and surname. I'm paying for a ProtonMail account linked to my name and surname. What prevents me from sending emails from my own server, using my own domain, my own logic and my own rules? There are so many ways to get back to me as a person in case of abuse, so the risk of spam should be low, right?
Unfortunately, filthy Microsoft thinks differently. Any email sent to @live.com, @outlook.com, @hotmail.com etc. gets sent back, blocked by Microsoft's spam filters. Even if it's sent through a legitimate paid domain name, from a legitimately purchased cloud service, through an account connected to a legitimate and paid ProtonMail account.
So eventually I've resorted to Mailgun to deliver my notifications.
Do you see the sad irony of this situation? Owning a domain, a cloud server and linking your account to a legitimate mail provider is no longer sufficient for you to deliver emails. If you don't want to use a Google/Microsoft/Apple account, then you have to pay extra bucks for a service like Mailgun, whose sole purpose is to do what SMTP servers have been able to do (for free) for the past 5 decades.
So, in a nutshell, running a mail service in 2021 is not difficult because it's hard to set up or it can't compete with the major players. It's difficult because a small mafia of cloud providers have done their best to make it difficult and force you either to use their services, or spend money for doing something that people have been doing for free for decades.
Email was supposed to be an open protocol, and barriers to set up your server used to be low. That's no longer the case. With the excuse of greater security and less spam, a small bunch of companies has built a criminal organization that has turned email into a closed standard, with Google and Microsoft alone running more than half of the traffic in US and Europe.
Shame on us all for giving up such an inalienable right of the Internet without even attempting a fight!
Do you have ~30,000 EUR / year for skilled admin? Plus ~20,000 EUR / year for hardware and other running costs? If you do, you can have your emails safely and reliably exchanged from your basement. If you don't, you can rent whatever on the 'net that suits your budget.
Though I'm not sure I'd recommend to invest into it. Back in the days that unix knowledge was valuable and setting up your own e-mail was a good way to learn thing or two. These days those skills are useless for most people, so I'd say use hosted mail and spend that time learning some more valuable skills.
Outlook blocks anybody that does not send enough mail, even if you've never sent any spam and are on a clean block. They're happy to let their users to send you spam, but ironically they still block you when you you try to report it to abuse. The good thing is they block you, so the email bounced and you know it wasn't delivered.
Gmail classify messages to a user who has never communicated with you before as spam. This is silent, so you never really know if an email to a Gmail box has been filtered out as spam or not. Their abuse inbox accepts messages but I'm not sure they do anything with it.
Basically, email has been hijacked by two companies.
I had assumed that the spammers had moved off to other mediums. Either that or they are specifically targeting big servers like Gmail and are leaving the smaller servers, with their varied (artisanal) anti-spam approaches alone.
Especially if your organization is a potential financial target.
A lot of legit emails end up in my spam box.
Another reason could be that I live in a country with very strict anti-spam and privacy laws (Canada). I have always assumed that spammers wouldn't care but who knows...