Please: I've experimented with this for over two decades, and continue to do so. I know that lots of people say that server-side filtering is fine. My direct personal experience is that for me it is not fine, including the spam filtering provided by Google's email service. I do not deny that for other people it's fine, I'm just saying ... don't deny my experience.
I regularly have emails to my GMail from colleagues end up in the spam folder. Perhaps as many as 5 or 10 per week, enough that I have to trawl through the spam folder every few days to make sure I've not missed something important.
Similarly, reasonably often when people reply to me, often after being prompted with a second email, or even a phone call, people tell me that emails from my "other" ISP to their GMail provided mailboxes have ended up in the spam folder.
It happens in waves ... it can be fine for months, and I start to think it's all good and I can trust it, and then there's a spate of emails going to spam when they really shouldn't. It's simply unreliable.
So for incoming email I do my own filtering. I can do nothing about my outgoing emails ending up in someone else's GMail spam folder except to be aware that it happens, and to keep track of when I'm expecting a reply. With some colleagues I've gone to a "positive acknowledge" system to preempt what have been expensive problems in the past.
* What's your DKIM/SPF/DMARC setup like? That all needs to be on point to get delivery to gmail to be reliable.
* Are you using a provider like Sendgrid, Mailchimp, etc. to send mail? Are you on a plan that gives you a dedicated IP address or is it part of their shared pool? The "it happens in waves" sounds a lot like a common problem with the shared pools where spammers send a bunch of mail through a service, the IP gets blacklisted, and it takes them a couple days to notice and remove it from the pool.
* do you have any "noisy" alerts or similar going out to people? Not necessarily spam, but eg, sometimes users getting frequent notifications, daily summaries, etc. will sometimes just flag them as "spam" rather bother to turn them off or manage them properly. At some point, that trains Gmail's filters to increase the spam likelihood on your entire domain.
* No, I'm sending email from my domain on my domain host. That has a single IP address shared among all their customers, but it's a very small ISP, and there are no spammers, and the IP address is not on any blocklists.
* No, I send standard email "by hand", and I know that people are not marking them as spam.
It's nice to see you go through these things because these are exactly the things I've gone through several times, so it serves to confirm that I've addressed the right things.
You say:
> ... look at the SPF/DKIM/DMARC related DNS entries with dig ...
I don't know how to do that.
> ... verify that they are correct.
I wouldn't know what it means for them to be correct.
As a further example, in trying to use postmark's service I get this:
In order for us to send you email digests,
you must set up a DMARC policy associated
with your domain. This allows us to collect
reports from the major ISPs about your DMARC
alignment.
Add this TXT record to your domain’s DNS at
_dmarc.solipsys.co.uk. with the following
data:
...
I really appreciate your feedback, and I'm passing it on to the people I pay to do this for me. The thing is, that's my ISP, and you tell me I shouldn't trust them.
> ... now you need to understand a lot about DNS and those mail related standards and do a significant amount of work to stay on top of the configuration to have a snowball's chance in hell of your mail getting through.
That's exactly what I pay someone else to do, and they are doing it correctly (as best I can tell).
So ... I just have to live with it. As far as I can tell, short of simply giving up entirely and using Google, I've done everything I can.
Is solipsys.co.uk your domain (from an earlier comment)? If so, I can see right away that there's no DMARC entry for it. That means that it's vulnerable to spoofing (I or any attacker could easily send an email to someone making it look like it came from your domain, eg, as part of a phishing attack). Gmail definitely penalizes domains without DMARC, making emails more likely to be classified as spam.
I am, for many reasons, so thank you for understanding that.
> If you want to keep believing that your ISP knows better than them, then I guess you're going to.
It's not that I believe my ISP knows better than Google, it's that I believe they are doing what they have been told is necessary.
> ... solipsys.co.uk ... no DMARC entry for it.
I will raise a ticket and get it dealt with, and as a part of that I can get the entire issue looked at more closely.
Thank you.
Based on what you've said here I don't know what you mean by "self-hosting". I have a package that I purchase from a company that provides me with my email and web hosting. I pay them, I configure my web client to connect using the details they provided, they deal with the configuration.
They are qualified in this field, they have certifications.
Everyone here seems to be using terms that obviously mean something to them than is different to what the terms imply to me. I have machines that connect to my email, hosted by the ISP with whom I have the contract. To me, that doesn't mean "self-hosted" ... I don't do any configuration of DKIM/SPF/DMARC/etc. That's all done by the people I pay.
So ... I really don't know what you're talking about. I suspect you know too much to be able to communicate effectively with me, who knows nothing. It's plausible that the field of discourse is simply too technical.