Gmail is a huge source of spam
drewdevault.com
drewdevault.com
> If you’re worried about mail deliverability issues, don’t be — it’s more or less a myth in $CURRENTYEAR. If you set up DKIM properly and unlist your IP address from the DNSBLs (a simple process), then your mails will get through.
Does this match most people's experience? Or is he just talking about the deliverability of smaller providers like Migadu? I run my own mail server and can never send emails to people on Gmail/Microsoft 365/etc unless it's in reply to an email that I received.
I've checked DKIM and the DNSBLs, and everything seems correct...is it just a matter of not having enough volume for email providers to trust me?
I'll see if I notice any configuration issues and report back to this thread.
Some feedback: you should add a PTR record (reverse DNS) for studer.dev and mail.studer.dev.
I also noted that your DMARC record has p=quarantine, which depending on the details of your outgoing setup, may cause your emails to be placed in the spam folder if DMARC fails for any reason. I generally advise against using DMARC with any policy other than `v=DMARC1; p=none` or else you're liable to run into issues, it's not a very good standard.
Otherwise, your configuration looks good to me. Also try https://www.mail-tester.com to see what they think.
I suspect it's probably a volume thing? As in I'm not sending enough email for Gmail to make any judgements about my mail server, since I'm only an individual. Of course there's not really a great way to resolve that...
Oh, and I get more and more false positives in gmails spam folder as well, and since they're deleted after 30 days I now have one more 'inbox' to monitor.
The problem the article discusses about needing to send lots of mail to qualify for these tools is because the systems don't scale infinitely - data storage still costs these firms money - and botnets mean basically every single IP on the internet will try to abuse your service at some point or another. So tracking deliverability data without any thresholding means tracking it for every connected machine in the world, which is hugely expensive, all to satisfy the tiny minority of people who feel passionately that they should run their own email server. The only possible justification for that would be recruiting related, but they don't have challenges recruiting.
If you have a few million of those or are part of our circle of buddies we just might cut you some slack. But otherwise, no matter that your vehicle performed fine just yesterday, it's up to us to change the rules at will and demand that you adapt.
Well what do you want for nothing? Randomly losing email is what the free email services do. That is hardly the fault of everyone else...
It does not. I used to work for a non-profit that ran our own SMTP server, and we had issues with our subscriber e-mails getting marked as spam, mainly for Office 365 users. We set up DKIM, DMARC, and SPF, but it still happened. We ended up having to move to using Google's SMTP servers just so our e-mails would go through.
- reverse ip. origin ip -> dns -> resolve to ip
- SPF
- DKIM
- DMARC
- Make sure ip are not on DNSBL
- Message ID
- Try to connect to gmail mx server using TLS
Then you have very low chance to get in spam unless you are sending spam and get blacklist. Time to time an gmail mx server will reject you, rate limiting, just try again with another and you will be ok.
Source: I run https://hanami.run and have to deal with this a lot due to nature of an email forwarding service that people usually use as one-off email (anything@domain.com for their one-off service) so it attract a lot of spam which I have to filter and make sure it won't go to gmail/microsft
I would say gmail is the best among big provide: Microsoft/Apple(iclouds) are garbage mail server where they just blindly trust DNSBL and block you no matter what. Gmail was way better.
However, if you use IPv6, it's a bit harder to avoid being flag as spam. It's really random and I don't know why yet and I contacted them and now "Waiting for 2 weeks to get a response"
To gmail point of view, they don't care about forwarding or replying to an email someone send you at all. All they care seems is about the message follow standard best practice(SPF/DKIM/DMARC/PTR) and send from a good IP. Sometime your IP can be flagged as spam wrongly on DNSBL. But it's very easy to get remove from DNSBL. Just submit a removal request, explain your situation and it's usually remove within a few days.
Also, the reason while your important emails are marked as spam is depended on gmail mx server you connected to. I don't know the detail but I have seen gmail mx server rejected/spammed my email, but on subsequent try if I connect to different one(by resolving other MX record), the email went through just fine.
If you send me an email vinh@hanami.run I can try to spot if anything may randomly flagged your email.
Also, did you have multiple SMTP servers?
I've checked everything on that list and it's all good, and I'm sending from IPv4. The really frustrating thing is that Google's own Postmaster Tools [0] doesn't want to tell me anything without "a sizable daily volume of email traffic (up to the order of hundreds)".
If it doesn't have to do with the origin IP's neighborhood, my only other guess is that I don't have enough email volume, which seems like a catch-22...
At the time I allocate an IP address, it's was listed on DNSBL. I tried like 30times to get a good IP and gave up. I instead just allocate a new IP, wait a week. Started to send email to myself every 15minutes to a gmail address. Anytime it isn't showed up in Important, I go in to move it to important. If it showed up as spam, I marked it as not spam.
After 2-3 weeks doing that, The IP is no longer listed on any blacklist and I started to roll them out.
Don't worry too much about IP blacklist. They aren't super important to gmail. If an IP stopped sending spam, they will be removed from IP blacklist eventually. Give it a week.
> "a sizable daily volume of email traffic (up to the order of hundreds)".
I send about 8,000 emails per day and somehow that Postmaster Tools not reports anything at all. I would say
If you send me an email vinh@hanami.run I can tell you what you can do to avoid being flagged at spam. Happy to jump in a 1-1 chat https://calendly.com/vinh-hanami/15min to help you on that.
(Fastmail, in case anyone is wondering. $50 a year, and there's nothing I want that they don't do.)
-we have never sent one unsolicited email
-100% of those emails are opened and read (I know because customers are activating their licenses). It's hard to imagine anyone mark the email with license they paid several hundred dollars for as spam
-we get tons of engagement, we answer around 30-40 emails a day
Something is terribly wrong with Gmail spam filter recently. Some of the emails from my customers land in my spam folder as well. They more often than not don't contain any links or anything out of the ordinary. Just a simple question or two.
It started happening a few months ago. It's a huge pain. All Google support tells us is that their magical box they can't disclose thinks our emails are spam even though there is nothing suspicious about either our website, the link or the content of the email. They can't tell us what triggers the filter cause it will make the job of the spammers easier.
I am now left to play the game against spammers: they try to imitate legit emails, if they get close enough then my emails are similar to spam, I have to modify them in random way, then spammers catch up etc.
Yay for machine learning I guess. Doesn't work so well even you have an adversary trying to fool you.
Btw, we use Gmail and it's Gmail spam filter which is causing problems. Google analytics is able to guess our monthly revenue and number of licenses pretty well. We send exactly as many emails. We are paying customer of Google Suite. It's still not their problem that their magic box puts our emails in spam but it's all on us.
Microsoft (Outlook.com) was particularly bad: mails were silently dropped without notice to me or the recipient even when I was replying to a mail they sent. I tried adding my e-mail address on my own servers to the trusted senders list on a MS account of my own and even then not all mail would go through.
Gmail ocasionally put me in spam but at least always delivered my messages.
At that point I gave up since I consider email to be too essential to be flaky and paid for Migadu. I have been very happy with their service and appreciate their pricing strategy (charging for number of mails sent, not the number of domains or mailboxes added). That said, I still dislike paying for hosting that I had sufficient capacity for on my own servers.
SPF, DKIM and proper DNS records and that is it.
You can test by sending email to a bunch of different mailboxes on different email providers, and then seeing which emails land. I've previously used GlockApps for this test and I've been happy with it, but you can probably find other similar services too. I'd be interested to know the results if you decide to test your deliverability.
> Our system has detected that this message is likely suspicious due to the very low reputation of sending domain. To best protect our users from spam, the message has been blocked. Please visit https://support.google.com/mail/answer/188131 for more information.
This message is extra confusing because mail from said "sending domain" is accepted by IPv4.
I use a gmail address and all my friends's self hosted mail have always ended up in spam by default
50% of spam come from a valid gmail account, as in, using gmail smtp server, come from gmail origin(.google.com reverse dns) have valid DKIM. The only way to filter out spam is look at the content and run some simple ML on it.
50% of spam come from non .com TLDs such as .cam .work from random servers around the world: China, Bulgari, Hungari, France, Germany...
But it's very easy to block other TLDs because their IP are on DNSBL already. But what can I do with gmail? Gmail IP get on DNSBL all the time.
Example: 209.85.222.196 is listed right now on spam.dnsbl.sorbs.net and ips.backscatterer.org
But at the same time, this is just the nature of an email service. If we have a large user base then no matter what we do, a small percentage of users are abuser which are successfully to send out spam using our services before we can detect and ban them.
On AWS SES, they requires user to keep a reputation of less than 0.1% emails flagged as spam. Go over than that and you will be put on a review queue. Reach 5% and get banned permanently.
But before that occurs, spam are send out already.
The abuse problem that an identity provider can solve is bulk account creation, but a) they are already doing a better job of it than any other free identity provider, b) given how low the numbers in the post are, this is clearly not any kind of bulk account issue.
(Also, not stating either the total number of suspensions or the proportion of suspensions that are for gmail seems like a conspicuous omission. The author states that 41% of all registrations are from gmail, and the absolute highest proportion of spammy registrations is 63%.)
The person who was banned was a troll, being rude to other users and posting nonsense to various mailing lists and bug trackers.
> I’ve forwarded many, many reports to abuse@gmail.com, but…
Not sure if it makes much difference but I’ve been using this form instead:
Wow, I'm looking at Google in new way now.
done