Hacker News Reader

Comment by thraxil | Hacker News Reader

A couple things I can think of:

* What's your DKIM/SPF/DMARC setup like? That all needs to be on point to get delivery to gmail to be reliable.

* Are you using a provider like Sendgrid, Mailchimp, etc. to send mail? Are you on a plan that gives you a dedicated IP address or is it part of their shared pool? The "it happens in waves" sounds a lot like a common problem with the shared pools where spammers send a bunch of mail through a service, the IP gets blacklisted, and it takes them a couple days to notice and remove it from the pool.

* do you have any "noisy" alerts or similar going out to people? Not necessarily spam, but eg, sometimes users getting frequent notifications, daily summaries, etc. will sometimes just flag them as "spam" rather bother to turn them off or manage them properly. At some point, that trains Gmail's filters to increase the spam likelihood on your entire domain.

View on HN

* I am assured by my ISP provider that the settings are as squeaky clean as they can be.

* No, I'm sending email from my domain on my domain host. That has a single IP address shared among all their customers, but it's a very small ISP, and there are no spammers, and the IP address is not on any blocklists.

* No, I send standard email "by hand", and I know that people are not marking them as spam.

It's nice to see you go through these things because these are exactly the things I've gone through several times, so it serves to confirm that I've addressed the right things.

You should be able to look at the SPF/DKIM/DMARC related DNS entries yourself with dig and verify that they are correct. I wouldn't trust an ISP's word, especially if you are experiencing problems. I would also recommend setting up weekly DMARC digests via Postmark's free service: https://dmarc.postmarkapp.com/ which should allow you to actually see whether emails are getting rejected and why and give you a lot more data about your actual SPF and DKIM alignment.

I'm not a computer person, certainly not a networks person, and I know nothing about any of these things, so everything you say might very well be true, but it amounts to days even to work out what you're saying. It's very likely you could talk me through it all, or I could embark on a course of finding out about these things, but I really just don't have time. I'm already over committed.

You say:

> ... look at the SPF/DKIM/DMARC related DNS entries with dig ...

I don't know how to do that.

> ... verify that they are correct.

I wouldn't know what it means for them to be correct.

As a further example, in trying to use postmark's service I get this:

    In order for us to send you email digests,
    you must set up a DMARC policy associated
    with your domain. This allows us to collect
    reports from the major ISPs about your DMARC
    alignment.

OK, that's fine. Then:

    Add this TXT record to your domain’s DNS at
    _dmarc.solipsys.co.uk. with the following
    data:

    ...

I have no idea how to do that, and suddenly I'm down the rabbit hole of things that other people think are obvious and easy, but leave me with no idea of what to do.

I really appreciate your feedback, and I'm passing it on to the people I pay to do this for me. The thing is, that's my ISP, and you tell me I shouldn't trust them.

Yeah, the thing is that if you have your own domain and want to send mail from it, that's really no longer the sort of thing where you can just connect to port 25, send mail and expect it to work. Spammers ruined that a long time ago and now you need to understand a lot about DNS and those mail related standards and do a significant amount of work to stay on top of the configuration to have a snowball's chance in hell of your mail getting through. It's not a Gmail thing, it's the entire landscape of email delivery. If you don't have the knowledge and time for that, you're much better off delegating it to Google, AWS SES, Sendgrid, Mailchimp, or some other service that takes care of it properly. If you're paying your ISP to handle sending mail for you and that mail's not getting through, you need to take it up with them.

I have passed this conversation to them, and they say that all the records and settings are correct. They say that the problem is the recipients' servers are marking the emails as spam, even though all the settings are correct, and the emails are clearly legitimate.

> ... now you need to understand a lot about DNS and those mail related standards and do a significant amount of work to stay on top of the configuration to have a snowball's chance in hell of your mail getting through.

That's exactly what I pay someone else to do, and they are doing it correctly (as best I can tell).

So ... I just have to live with it. As far as I can tell, short of simply giving up entirely and using Google, I've done everything I can.

You seem pretty set on staying with your ISP and I'm not going to try to convince you otherwise, but "we're doing things right and Gmail is wrong about how to process email" wouldn't carry much weight for me. I'm 100% sure that the Gmail devs know more about email delivery to Gmail addresses than I do and 99.99% sure that they know more about it than your ISP. If you want to keep believing that your ISP knows better than them, then I guess you're going to.

Is solipsys.co.uk your domain (from an earlier comment)? If so, I can see right away that there's no DMARC entry for it. That means that it's vulnerable to spoofing (I or any attacker could easily send an email to someone making it look like it came from your domain, eg, as part of a phishing attack). Gmail definitely penalizes domains without DMARC, making emails more likely to be classified as spam.

> You seem pretty set on staying with your ISP ...

I am, for many reasons, so thank you for understanding that.

> If you want to keep believing that your ISP knows better than them, then I guess you're going to.

It's not that I believe my ISP knows better than Google, it's that I believe they are doing what they have been told is necessary.

> ... solipsys.co.uk ... no DMARC entry for it.

I will raise a ticket and get it dealt with, and as a part of that I can get the entire issue looked at more closely.

Thank you.

Ah, so you are self hosting your email! That's where the trouble lies then. I wouldn't do that myself, too much headache. Just use Gandi's webmail.

I suspect you are using terms that to you are obvious, but to me are technical terms that don't mean what they seem to mean based on the words involved.

Based on what you've said here I don't know what you mean by "self-hosting". I have a package that I purchase from a company that provides me with my email and web hosting. I pay them, I configure my web client to connect using the details they provided, they deal with the configuration.

They are qualified in this field, they have certifications.

Everyone here seems to be using terms that obviously mean something to them than is different to what the terms imply to me. I have machines that connect to my email, hosted by the ISP with whom I have the contract. To me, that doesn't mean "self-hosted" ... I don't do any configuration of DKIM/SPF/DMARC/etc. That's all done by the people I pay.

So ... I really don't know what you're talking about. I suspect you know too much to be able to communicate effectively with me, who knows nothing. It's plausible that the field of discourse is simply too technical.

Reply on news.ycombinator.com