Sorry if this guy's "audience building" business is affected, but as an end user not worried about "my open rate", I appreciate Google's aggressive filtering and urge them not to let up.
The thing is, I run my own personal email server. It's not used for spam ever. It's only personal email to family members, friends, and the odd reach out to support. The volume of email is so small it's ridiculous. I might send up to 10 mail (1 receiver each) per month. I set it up with SPF and DKIM properly, triple checked.
And yet, for whatever reason, sometimes my email end in spam. Generally when I send attachment (like pictures). But sometimes it's just completely arbitrary.
How about this : for you it might be a "few false positive", because so much of the wanted mail traffic originates from google in the first place. But for the few of us that use mail as was intended, this becomes a huge problem.
Having a mailing list doesn't necessarely mean the content is unwanted. I'm subscribed to HN replies for instance. I want those email. If it started going to spam, I might consider "hey it's a few false positive". But for the HN replies guys, it would be maddening to be unable to fix the problem because Google can't be arsed to have a support channel.
From my POV, Google is way past "aggressive filtering" territory and way into "monopolistic behavior".
I know a few people whose professional work involves a lot of delivering explicitly-requested, non-spam emails. With most email providers, they don't go to spam because they aren't spam. If they get flagged as spam servers (usually right after spinning up an instance, when the confidence is low), they can contact someone and get it sorted out.
With Gmail? They have extensive lists of tips and tricks to "sneak through" emails that their users specifically asked for. They 'prime' servers with extra-safe traffic to avoid getting frozen in the low-confidence early emails. And they still regularly run into problems. They just don't have the size to be trusted by Google, or to get business contacts to solve the problems that do arise.
It's a pretty concerning state of affairs - Gmail will deliver anything sometimes, but it seems to be a very high bar (vastly higher than "not spam") to ensure you don't regularly drop messages.
The idea that their spam filter is particularly good is ridiculous too. I get more spam to my gmail accounts than to my self hosted mail accounts with greylisting and spamassassin.
Except... Google claims >90% of spam never even reaches your Spam box, only the ambiguous messages. So why do I get 10x the spam at Gmail as at any other account? Are they not actually good at detection? Are they just a high value target that gets hit more often?
Or (my guess) are they trying to get clever? Google Doc's spellcheck is infamously bad because they're trying to use machine learning approaches without having a simple English dictionary as a fallback. Is Gmail trying to detect solely on email sender/content and totally abandoning approaches like greylisting?
Don't do that! Emails should be instant, not with stupid greylisting delays.
Nothing in the RCFs says emails are supposed to be instant. Reliable, yes. Instant, no.
First, greylisting is HIGHLY effective. Something like 95% of email connections are spam. Almost all of them fail without ever hitting disk, via greylisting. If they actually retry, then the chance that the sender IP or email content is in an RBL is MUCH higher, so spam that actually retries will be more likely to be rejected at HELO by an RBL or filtered after it is accepted. The incidence of false positives with grey listing is VERY VERY low. Using greylisting means that very few emails need to be accepted and then filtered. Those are the false positives that really hurt. The sender receives no bounce, and the recipient probably doesn't look at his spam folder every day, especially when it is full of hundreds of junk emails. Greylisting makes email more reliable.
In addition, better greylisting systems note senders that reliably retry and automatically whitelist them. My mail server actually delays less than 2% of legitimate emails due to greylisting.
e.g. https://pastebin.com/u48DAaLP
That particular example was sent via SMTP, but I had the same problem when sending via the Gmail web interface, and it occurred sending to at least three different Google Apps domains.
After I moved my domain off Google Apps (I switched to Fastmail for a variety of reasons, but that issue was the kicker), I was able to send to those same addresses without issue. In fairness to Google, I was on the Google Apps free tier at the time, so there was nowhere to go for support.
I never understood what was causing these bounces. The entire email chain occurred within Google's control, passed SPF and DKIM checks, my domain isn't on any blacklists nor used for spam that I know of, and isn't used for anything but personal emails. And I've had it registered for years. Nor was it the content of the message. I had a friend with another Google Apps domain setup an alias for testing and when I mailed that address it bounced the same way, regardless of the message body. Emails to other addresses in his domain were not marked as spam... it was only Groups aliases that bounced.
I definitely felt that gmail was being too aggressive on this too. I sent a bunch of detailed replies to middle/high schoolers working on science projects when they reached out through my public-education webpage and when they went to gmail I never got a reply but if they went to some other server, the conversations usually continued nicely. That was pretty frustrating.
It's probably your IP address.
Sure, SPF and DKIM say you are allowed to send these emails, you are who you say you are, etc, but you also aren't a Sendgrid IP address (or someone similar) so you're treated as less trustworthy.
I could be wrong, but I believe many email servers take these things into consideration (please let me know if I'm mistaken!)
Many spam filtering systems work on a 'points' system, so being on the dynamic IP list adds a lot of points, and if you also have other things that add points (being a mailing list, having 'unsubscribe' links, attaching images, having links) it might be enough to put it over the edge and get marked as spam.
You can do a lookup [3] to see if you're on any of these lists.
[1] http://www.sorbs.net/faq/dul.shtml
I run my email through google for my domain, and the IP assigned to my mail server is actively used to send spam (like, daily active reports on SORBS), so I can't even fill out their form to get it unblocked.
As a result, I need to email my bank from a different email address. So stupid.
Well, here's your problem. Have you thought about self hosting?
Check your reverse DNS as well. It helps if it matches the HELO advertised by your mail server.
I'm not sure this argument is related to the rest of your post, but FWIW, GMail/Inbox puts emails from mail lists into their own "Forums" category, not in the spam folder.
Running your own email ain't a hobbyist thing anymore, you gotta play like you're in the NBA. I'm doing all the above, my deliverability is solid. And pay attention to the changing landscape - for example, I will be implementing ARC real soon.
And the postmaster tools don't work for me on google because the volume of email I send is too low. So there's that.
On a sidenote : It's about time something like ARC comes out. If I understand it correctly, it should fix many setups that involve mail forwarding. That'd be a godsend for me, I'll probably set it up soon enough.
DMARC is not a nuclear option, no-one starts out with the reject policy, in fact most of my domains are still p=none and I monitor the reports.
You absolutely can and should - irrespective of how much information it gives you in return - verify your domains in Google's Postmaster Tools, monitor your IP addresses and junk mail tagging at Hotmail via Microsoft SNDS/JMRP, and setup DMARC reporting e.g. using Postmark's weekly digest.
I strongly suspect your lack of TLS is hurting you in particular. For this and other reasons besides I advise you to enable TLS for all inbound and outbound. And use a real cert.
I'd disagree. I've been running my own just fine for years. Yeah, you have to put some work into it, but spf/dkim/dns have been suiting me just fine.
It's overrated. And like mentioned in this article: it makes regular email delivery unreliable.
For me FastMail has largely replaced gmail. I'm much more happy with them than I ever was with gmail.
And they have an actual support channel.
Edit: the "gmail is the only real option" meme is getting pretty tired. They were clearly the best option 10 years ago, but times have changed. No need to repeat old truisms.
Google's solution is necessary because bad actors in email abound.
Literally just started to switch back today from Fastmail
> since they've pretty much solved the spam problem for me
#1 reason. I tried Fastmail for a month or two but I've only just hit the 200 spam messages to train my "personal" spam filter, all of them "greetings of the day" website/SEO related. Realised I've been fighting to get to this personal filter level and then when it enabled.. I received another spam message.
I'm out. Love the idea of not relying on Google for everything but I guess for now spam filtering is just one of those things Google does good.
Also note that you can tweak your spam preferences. For example, I have mine set to mark things at spam at a score of 4.0 instead of the default score of 5.0.
There are other things I missed from Inbox too such as reminders "hit t - set reminder", snoozing emails till another time and I have a few people that contact me via Hangouts so I ended up loading Inbox occasionally anyway to see those messages.
No fault on Fastmail really, if I'd never discovered Inbox I'd still be using it. In regards to spam I just can't be dealing with training a spam filter, I was reminded it was one of the reasons I switched from self hosted to Gmail in the first place.
Edit: For what it's worth the SEO and website spam emails weren't getting a spam rating at all, most were <2.0
Edit 2: Also while looking at the spambox for that last message I now realise a lot of my server backup status messages were going into the spam box with a 7.1 spam rating.
Edit 3: I'm going to stop editing after this one. Edit 2 was on me, the backup messages were sending direct rather than via mailgun.
And the very subject of this article is emails that people want to receive getting filed into spam by Gmail (but not by other mail hosts).
Dovecot + Thunderbird even gives you client editing of server-side filtering based on all sorts of criteria, and self-hosted webmail (e.g. MailBox) is pretty good these days as well for when you're on the go.
That's what I've been using for my two primary domains (vanity personal and business one) since 2004 and 2006. Those filters work pretty well. I can't compare with Google's one but they're enough. I prefer to get some extra spam than losing a business email and maybe a new customer.
Maybe there is a way to keep using Gmail, read from an external POP3/IMAP mailbox and disable Google's antispam on messages fetched from there.
The main reason you need a spam-filter is because your address ends with @gmail.com. Man I've lost important mails due to gmail, I'd never use it for anything remotely serious. Works fine for interactions within google-services though.
But apart from that there is no decent client and their notification support is not reliable either, nor can you rely on their spam-filter. Why even bother with email if you are going to use gmail?
Fastmail on the other hand only classifies spam as spam (with way less than one spam mail per month getting through) which for me is far superior to Google's "I have to check the spam folder regularly because anything can end up there"
Because is about those ones the article is about.