Comment by apocalyptic0n3 | Hacker News Reader

Comment by apocalyptic0n3 | Hacker News Reader

Parent Full thread

Only tangentially related, but I could use some help. We have a nearly 10 year old newsletter service that was built long before I was hired. It's all self hosted and we're getting abysmal delivery rates now (~60%...). We're only sending about 5000 emails a week.

We want to offload, at the very least, the sending onto a third party service where our delivery rates won't be so bad. The trick here is that we send the newsletters on behalf of clients that we have built sites for since we launched our newsletter service and they currently send with a FROM and a REPLY-TO of the domain we are sending for. Obviously, this does not mesh well with current spam practices. Third party services usually require you to verify the domain, but the vast majority of our clients do not have access to anything related to their domain or DNS and it's not realistic to move to SES, MailChimp, SendGrid, etc for that reason.

Does anyone have any suggestions for how to approach fixing this? I've already asked about sending out from client-name@ourdomain.com and having a reply-to of the client's email address, but was shot down for apparent contractual reasons.

>It's all self hosted

>Third party services usually require you to verify the domain, but the vast majority of our clients do not have access to anything related to their domain or DNS

Do you see the contradiction here?

If you are sending it from your domain there is no issue migrating to mailchimp.

We spoof the client's email. So while we are sending from our IP address, the FROM is actually john@clientllc.com, the FROM name is John, and the REPLY-TO is john@clientllc.com.

10 years ago, that passed. Nowadays, it's unacceptable. I'm wondering if anyone has a solution that doesn't involve any DNS changes for the clients we are spoofing, because it just won't happen, unfortunately. Our entire newsletter is built upon email spoofing, which is a terrible practice but I cannot get management to allocate any more resources to fixing it than "just make it work again". I'm going to try to push sending out from our (verified) domain again using SMTP (so we keep our system other than the sending part), but I'm doubtful I will get the approval for that.

I see the contradiction and realize that the verification process is in place for an extremely good reason, but actually implementing a fix is not as simple as you would hope. I was just hoping for someone to have an idea that I had not already thought of.

You're not going to be able to send from the clients email if you're unable (unwilling?) to implement the DNS changes required to verify the domain. Surely that's the minimum required to fulfil "just make it work again"?

This is what I suspected, but I felt it worth throwing it out there and seeing if anything is possible. We're doing things the way you did 15 years ago and that just doesn't fly anymore.

What about using no-reply@service.com as FROM and client's email as REPLY-TO? Spam detection gives much more leeway on REPLY-TO address.

I've suggested that as well. I'll bring that up again, though, when I'm talking to management about it again. Our bounce rate jumped from 25% to almost 45% in the last month, so it may be enough of a shock to make some changes.

I know that you said but I'm afraid a major part of a spam scoring is coming from DNS related checks (SPF, Domain Keys...). I would concentrate on that... offer yourselves to configure that on behalf of your customers, get those DNS things shorted out and your delivery rate will get higher.

In more than one industry I work with it's nigh on impossible to gain access to DNS records for bulk mailing purposes. Third parties may hold control over domains, and your emailer may not be important enough to get someone to actually put in a work order to get records changed etc etc.

Not always easy to fix the human side.

Yeah, like I said, I realize that's how spam scoring works and that our methods are just extremely outdated now. Problem is that our clients frequently do not have access to their DNS and nothing has changed on the DNS in 5-10 years. The purchased a site and email service a decade ago and they won't think about it again for another decade.

> our clients frequently do not have access to their DNS

How could they not have access to a service that they pay for?

It sounds like you have a human problem on your hands, not a technical one. I would think about how to get an updated SPF into their DNS. If the boss said "just make it work" then to me that means to call the clients and hand hold them through figuring out who hosts their DNS records and how to open a support ticket or reset their self-service web portal login password, or whatever else the client needs to do to get those records in there.

SPF is very easy to configure and will improve your deliverability significantly. After you configure SPF, mail servers will acknowledge your servers as a valid sender for your client domain. It will no longer be spoofing. The only difference between spoofing and not spoofing is domain validation.

DKIM is not tied to an IP address but the 2 extra steps of complexity involved with DKIM have allowed for SPF to be the dominant domain verification method for sending email.

> How could they not have access to a service that they pay for?

Generally, they are on autopay and haven't had to sign in for several years. We often run into an issue where the person who set it up is no longer with the company and they no longer have the associated email address set up, so they can't do a password reset (without setting up the email, at least, which means hunting down how to do that; we're talking <5 person companies that haven't had a new employee in years)

> It sounds like you have a human problem on your hands, not a technical one.

This is definitely the problem. :/

> It sounds like you have a human problem on your hands, not a technical one. I would think about how to get an updated SPF into their DNS. If the boss said "just make it work" then to me that means to call the clients and hand hold them through figuring out who hosts their DNS records and how to open a support ticket or reset their self-service web portal login password, or whatever else the client needs to do to get those records in there.

Exactly my point

As gardnr says, you need to solve this problem by involving the clients. Trying to fix this without some DNS fiddling is going to waste your time and you'll have nothing to show for it.

The world changes, and adaptation requires maintenance. 10 years ago is almost half the age of the web ago. This is more a business decision - do you let those accounts rot - than it is a technical decision. Because the business decision needs to include support time for user interaction.

If it works for your clients you could always set up some email relay sort of deal on your emails? Get them subdomains on your domain name and relay mail sent there to them? Then you could do the dns on your own servers.

Try slowing down your outbound sending. I used to manage a mail system that generated a lot of outbound mail, and speed of the mailspool was a huge issue for the big consumer services.

Prove it with a small example that you get better rates with DNS fixes and your management will think about it differently.

Have you tried spoofing the envelope address or reply to address, but have the header address as one you control?

You send from other people's domains without setting up DKIM/SPF? You're fighting a huge uphill battle if that is the case. I would highly recommend sending from a domain you can verify and then using the reply-to field for responses. It isn't perfect, but your deliverability rates will sky rocket.

Can you even get emails to Yahoo addresses?

Ultimately you're not going to get the best deliverability unless you have access to the DNS records you are sending from, because you won't be able to pass DMARC.

However, you can still pass SPF and DKIM checks by using your own domain, which involves taking advantage of SMTP's envelope sender address. This can get you acceptable deliverability, assuming good IP, non-spammy content, low volume sending, bounce management etc.

There's a pretty good StackExchange answer here that explains the differences between SMTP envelope and message aspects in more detail: https://security.stackexchange.com/questions/30732/why-is-it....

There's also a good MSDN blog post series on the various email authentication techniques and how to get them working: https://blogs.msdn.microsoft.com/tzink/2013/04/24/how-to-set...

The gist: set up an address, and SPF and DKIM on a domain you control, and use that address as the SMTP envelope sender. Then "spoof" the FROM header.

The caveats: this technique is still used by some spammers; some clients will display "From: (your domain email) on behalf of (actual sender)"; some MTAs use SenderID which applies SPF to the FROM header's domain; some (but not most) providers almost need DMARC to get bulk email to the inbox nowadays.

Reply on news.ycombinator.com