Hacker News Reader

Comment by wpietri | Hacker News Reader

Agreed! I've been running my own mail server for 20+ years, and I would not recommend that anybody do this on a lark. Deliverability to GMail is a huge problem for me and has been for years. I even asked friends at Google to find out what the story was. All I got back is that the mail group was so careful/paranoid it wouldn't talk to them either.

I would long ago have switched over to a vendor, but I use qmail-style tagging for sub-addresses (that is, instead of user+sub@domain, I use user-sub@domain). Almost nobody supports that. Especially not GMail (where my friends could tell me that was tagged as WONTFIX).

If anybody has solved this, please do let me know (on here, via Twitter, or the email address in my bio). It's maddening.

View on HN

Funny. I'm somewhat new to the game but my email server has had very few problems delivering anywhere (hosting a small business' email, and personal email for the employees.) I've got spf, dkim, and dmarc. I use tls on everything. Got a static ip (linode) with rdns. I've never had a problem delivering to gmail. For a while i had problems delivering to Hotmail/outlook, but participating in some sort of junk mail reporting program (JMRP i think it was) fixed that issue. Now, if i have deliverability problems, it's cause something broke, or my tls cert expired, or i added a new ip that the email server decided to start using. (Sorry for formatting, on mobile)

Sounds like you have the same setup I have.

I never get bounces from Google. And after the first day or two of removing yourself from 2 or 3 blacklists, its great.

I use mailcow FWIW

Which mailserver are you using? I'd love to try hosting my own email.

I would advice to try with mailinabox.email . It is very opinionated, which, for a beginner, is a good thing with mail.

I'd encourage everyone to try this on a tiny droplet or linode with an unimportant domain (don't just migrate your entire companies' exchange into it on a Friday afternoon). It takes you an hour tops, after which you can poke around and see all the moving parts that make a good mailserver tick.

You'll also be contributing to a stronger, more resilient internet, by making it a tiny bit more decentralized.

Where your IP came from? More specifically, who is the IP block owner? Ana ISP? A cloud provider, like AWS?

That's been my experience. I've been running a server since 2012. I've had more problem with incoming mail - specifically being unable to whitelist certain Yahoo servers which get killed by SORB and other spam trackers - than with sending email.

Having run my own server for 20 years, my experience is also that it is relatively hassle free. I also run my own DNS and have a static IP, which might help with keeping email related DNS records up-to-date.

Running the server is generally "set and forget". Every few years an issue might pop up that requires attention. These issues are generally due to a tightening of other servers' requirements rather than an actual technical issue. When such things do occur, symptoms are an occasional email rejection, and a bit of digging reveals the cause and fixing the cause returns things to normal. I've never had a wholesale rejection of mail from my server

For example, when SPF and DKIM came in, I had to add those records to my DNS. When Let's Encrypt came on-line I proactively added TLS to the server. A number of years ago lack of a Reverse DNS record got me on a blacklist for a short time. That was fixed by contacting the IP address issuer (my ISP) and getting them to add a reverse DNS record. Just make sure the reverse DNS hostname matches the hostname that your email server uses in its HELO messages. A week or so later I was automatically off the blacklist and the few rejections went away. I've never bothered with DMARC. This is the sum total of my experience with running the server.

At times there have been physical problems with the server or network outages with my ISP, but I discount these on the basis that it is my decision to run a server on a desktop PC in my house rather than in a data centre. Easily fixed if I wanted to throw money at it.

I've been running my own E-mail servers for the last 20 years or so, and have not seen these delivery problems.

I've had a similar experience.

Setting everything up in the first place was a massive pain in the arse, definitely not for the feint hearted (I imagine there are tools/scripts to make it much easier nowadays), but it's been almost plain sailing since.

It's very rare that I have delivery problems, maybe once every year or two, which is roughly the same as through my O365 mailboxes!

Only issue is when there is a delivery problem, there is usually nothing you can do. Some of the block lists have a procedure for removal, many don't.

But after running my own mail server for something like 15 years, I've decided I just don't want the hassle any more (even if it isn't much work), and plan to move everything to O365.

I run my own mail server for receiving, but send through a commercial provider (Fastmail). I also use user-sub@domain style email addresses. If I have occasionally have to send email from those addresses, I edit the From line in the Fastmail web interface or my email client.

I don't have a problem setting up a full email server, but I don't want to babysit it for just one person using it. I have too many real life obligations these days to try and fix the mess if a problem would arise.

Interesting! Maybe I should try doing the outbound via Fastmail or some other provider. I couldn't shut down my mail server entirely, but it might solve my deliverability issue.

I use postmark to do my delivery. They support outbound smtp that my internal postfix relays to and you pay per email even tho they gave me so many free credits I havent paid anything.

I was thinking about doing the same with mailgun. Do you have any obvious delivery issues with outgoing mail marked as spam?

Curious why you're so attached to the qmail style subaddressing. Personal preference or is there a technical advantage?

Possibly not getting blocked. Many sign-up fields will block foo+spam@bar.net if I try to use it.

I've ran into this issue maybe twice in years and I'm always using that for websites. I'm having a feeling that issue is a bit overblown.

I just add . in arbitrary places to distinguish sources. Since there's no telling what dots a gmail user thinks are a significant part of their address.. (plus I have seen fail and late enough to be a pain, but due to http parameter escaping, etc, rather than intentional fitering.)

That's why I use "." as the separator.

Now try using an email from a domain name with more that 3 letters in the TLD.

Almost no one considers that a valid address.

So if I bought my “name.rocks” I’ll have trouble using that as an email? I was looking forward to “firstname@lastname.rocks” and I never even considered it would be a problem. I’m disappointed

I have something similar - I haven't had many problems with the not being accepted online. The few times I've had issues are with old point-of-sales or appointment booking systems.

I have a .help domain for handing out to services and it works without issue.

Yes. I'd say way more than half of sites won't accept that as a valid email.

And you'll be flagged for fraud as well.

It's very frustrating.

Probably because s/he has those addresses out there in the wild starting from a time that it was still reasonable to run an email server, so like 10+ years and probably a bit more.

That's exactly the case for me, but I've been handing out these addresses for nearly 20 years. I also suffer from the problems with gmail that the GP referred to. I can't even forward all my email to myself without having it dumped into gmail's trashbin!

I've given out hundreds and hundreds of them. Almost any time I fill in an email address on a form, I'll tag it so I know where spammers and other miscreants get their addresses from. As have a number of friends I host mail for. Swapping to a mail provider that doesn't support them would be a giant pain for all concerned.

I use Migadu.com for my email and they allow you to create regex based catchalls [0]. I have one that accepts every email address that begins with "cat". it makes it dead simple to create a fresh email for new services.

[0] https://www.migadu.com/en/benefits.html#anchor_catchalls

I don't have any problems with Gmail, but there is no way to know why :)

As far as I know the most important factor for deliverability is your IP address: my /24 have been clean for many years because the network operator actually respond to abuse@ reports.

If you have problems my first advice (aside from checking that SPF+DKIM+DMARC+PTR+banner+etc... are OK) would be to find a better ISP.

This is a physical server (at Datacate [1] in Santa Clara), so switching ISPs is not a small undertaking, especially given that I'm only using 1U of space. Last I looked the netblock reputation seemed fine, so I'd only be inclined to switch if I had clear evidence that was the real problem.

[1] in this block if it matters: https://rdap.arin.net/registry/ip/198.167.232.0

Yeah, the IP address of your MX looks clean indeed, and your config looks good too... I wonder, have you ever tried sending from @scissor.com instead of @williampietri.com?

I definitely observe the problem on both domains. Maybe less on my name domain, but it's hard to tell.

+1 about the opaqueness of gmails spam filters. I once had an email from a google recruiter (@google address) go to spam in gmail so at least it's somewhat egalitarian (that was 5+ years ago).

same thing. Had an email from a recruiter and another from financial go to SPAM in gmail.

More, google knew this was a thing because they warned me.

That makes sense though. An aggressive recruiter overusing their work email to contact cold leads will get flagged a lot, even if the domain is clean I imagine they also train based off of individual addresses.

Yahoo's disposable addresses are constructed in that form (basename-keyword@yahoo.com), but I'm not sure if they support them on business/custom domain accounts.

(The base name also has to be different from your real email address, but that's a fairly desirable feature for disposable addresses since it doesn't reveal the real address like the '+' suffix on Gmail does.)

Just to be clear: the plus(+) thing is not a gmail thing, it's part of the RFC for mail-addressing. A standard.

I don't see why sub-addresses would be a problem with regards to sending mail. Isn't it wholly up to the receiver to interpret the username part of the address (or not)?

It's not. It's a problem for me just paying somebody to host my mail entirely, as most vendors don't support it.

I do two things:

1. Send mail via a static IP (that I pay extra for)

2. Send all mail through a reputable 'smarthost' (mine is provided by my ISP as part of the 'business' package).

This fixes all sending mail problems. For extra fairy-dust, I added SPF records to my mail server IP.

The BIG problem is Spam. Unless you pay for an intermediary spam filtering service, you will NEVER be on top of it. After self-hosting my own mail system for decades, I am now seriously planning a migration to Office 365.

> The BIG problem is Spam.

YMMV, but a combination of graylisting, RBL and "sender address valiation" tends to drop inbound spam rate to nearly zero.

I'm currently hosting a spamassassin instance, and route incoming mail via that, however a lot still gets through :(

I'm using SA as well; it's fine for me. My big trick is to route a bunch of stuff right into the spam training system. E.g., when a tagged address I've given out is compromised, that goes right to training. The same goes for spam to random names: also right to SA spam training.

My theory here is that statistically, by the time a spammer tries to send one to an active address, they're likely to have already fed things into the trainer, often multiple times.

That is such an overblown issue unless you're coming from a spammer network or trying to use something like a VPS to send mail.

I switched over to Gsuite and I use qmail style addresses, it’s not as friendly as using + addressing (as in it just works) but you can use regexp rewrites on the paid versions of Gsuite to fake it.

(Disclosure: xoogler, but still happy with Gsuite...)

> I've been running my own mail server for 20+ years ...

Out of curiosity, where does your MTA live? Do you have a VPS? VM/instance at a cloud provider? ISP with static IP? Other?

I have a physical server colocated with Datacate in Santa Clara. As far as I can tell the address space is good, and I've had the server there for a number of years.

Wouldn't tagging only affect inbound mail? I would've thought a domain forwarder would work around this for migration?

Fastmail allows this, if I’m understanding your need correctly. I’ve been using it for years.

You mean you can send emails to <user>-<tag>@<domain> and it'll be delivered to <user>@<domain> account (assuming MX for <domain> is pointing at fastmail)?

Is there anything you need to enable to allow this? I just tried, and the email got bounced back.

Or are you talking about email aliases? As in, you register <user>-<tag>@<domain> first before sending emails to that address?

This is true! But when I last talked with them, I couldn't figure out a disaster recovery plan. They had no way for me to back up my users' mail aside from IMAP syncing. Which means I'd have to know my users' passwords. Which seemed slightly more ridiculous than just continuing to host my own mail.

Two options:

1. share everyone's mailboxes to an admin user who does the backup via IMAP.

2. create an app-password for each user which can be used to backup that user. This can be done as an admin user on your account.

Neither of them require knowing the user's password - an admin can override into each account unless it's specifically locked down to deny that. It does require a separate app password per account, we don't have a way to create a single password which can view each user's account without them explicitly sharing the folders, but I'm not sure how you reasonably do anything else without it being a backdoor behind all the privacy settings on each account.

Ah, interesting! Yes, I'd be fine with either of those. Right now I'm just backing up via rsync of the maildir tree, so it's not like these are any worse. I just didn't want to a) know my user's passwords, and b) have to have them give it to me any time they changed it.

Thanks for the tip. Maybe my Christmas present to myself will be to murder my mailserver!

Sorry - busy at IETF this week, didn't get to reply before. I would recommend the per-user app passwords over sharing all the folders to an admin account - because that way you see the \Seen flags as the user when backing up.

Check out uberspace.de. They are great and use qmail.

For what it's worth, I don't even need to use qmail. I've been using Postfix for years, which allows this as a configuration option. But I'll check them out for sure.

If nobody supports it then why did you choose it?

This sort of snotty reply is one of the reasons I discourage people from using HN.

I chose it 20+ years ago because that's what qmail supported, back when subaddresing was a new idea. Over the ensuing years, most people ended up converging on + as the more common option, but that's only convention, not a standard. And what standard exists wasn't written until 2008: https://tools.https://tools.ietf.org/html/rfc5233ietf.org/ht...

If you took that as snotty then that's on you. Maybe people who can't take every simple question not being a delicately delivered paragraph full of qualifiers should be discouraged from using HN, yeah.

Yes, please continue to blame readers for your words landing badly. That will surely work out well for you.

Reply on news.ycombinator.com