Learn why a message ended up in your spam folder | Hacker News Reader

Learn why a message ended up in your spam folder

gmailblog.blogspot.com

112 pointscleverjake27 comments

We've been setting up email infrastructure using sendgrid on a dedicated IP. The domain it's emailing to is a new domain as well. We've got that domain set up with google apps and use it to send email to people without issue.

We did a mailing to a few select people on our new mail server. For the 20 or so people using gmail that received the mailing, those who had never received an email from colin @ newdomain had the email go to spam. Those who had received a message from colin @ newdomain had the email (from a different user) in their inbox. Anecdotally, it seems gmail highly values a previous email from that domain, but only for that one recipient.

Email is tricky. We're sorting through the issues of sending from a new IP and new domain. We're learning that email sending is like a credit score. No score is as bad as a crappy one.

Email is a headache! Here are a couple tools I use to verify SPF records, DKIM records, etc:

http://www.openspf.org/Tools and http://www.kitterman.com/spf/validate.html and http://dkimcore.org/tools/dkimrecordcheck.html

Also, you might learn about warming up an IP address for mail sending if you're going to be pushing volume: http://docs.sendgrid.com/documentation/get-started/sending-p...

Sendgrid's reputation checking and interface for seeing and clearing bounces, spam reports, etc. are extremely useful too.

I imagine you already know all this, but others may not :)

In our case, gmail was deciding the email was spam. As a rsult it doesnt get reported to sendgrid. It makes sense from googles point of view. If they reported it to the sender it would be easier for spammers to learn how to work around it.

> If they reported it to the sender it would be easier for spammers to learn how to work around it.

Of course, it would also mean those of us sending legitimate e-mails that get canned by these ever-more-aggressive spam filters know that someone did not receive the information they asked us for so we can do something about it.

Silent failure breaks e-mail. It is irresponsible, and the damage due to not getting a legitimate message through can be far greater than the damage due to letting a spam through (which you wouldn't be anyway, the spammer would just know they're not making it).

Make sure you set up your SPF record. It'll look something like:

  "v=spf1 a mx include:sendgrid.net ~all"

You can check your current one with:

  host -t txt yourdomain.com

Yours will likely differ based on your needs, but you need the include: for SG there.

Send an email through SG to a Gmail address and check the full headers. Look for spf=pass and that they're attaching all the DKIM info.

Now this blog template does not even produce content via Google cache or text-only cache, when Javascript is disabled. I guess I'm done reading items presented in the newer Blogger templates.

It is also totally unusable on mobile: If you swipe to the side (a common action to check if the page is displayed at fit-width zoom level) , a different article loads, because someone at Google can't tell the difference between a blog and a book.

I deeply regret allowing greader on android to update to a version that (unknown to me) contained that "feature."

Oh my god, what an horrendous design, it made my poor old PC cringe in despair with all the javascript to load!

I went to my SPAM folder, all excited, hoping to find some cool math/programming stuff, but it was just a plain boring message: Its content is generally found in Spam messages. Ah... that's a downer...

I was excited to see this since just last Friday about half (~2500) of my Gmail subscribers didn't get my last Hacker Newsletter issue since it went straight into their spam folder, but it doesn't seem to provide enough information to really learn why it ended up there.

I think it would be a tough to balance between providing useful information to users and not revealing too much to spammers.

It looks like from http://support.google.com/mail/bin/answer.py?hl=en&answe... they probably reveal the message fell into one of those five categories.

FWIW I received last Friday's with both DKIM + SPF intact with a PASS.

Maybe your DNS was unavailale for a short period of time during the delivery?

Not sure, I guess that could of been the case? I need to look into it more. Thanks for the info!

"It's similar to messages that were detected by our spam filters."

How is that helpful to anyone?

Looking in my own gmail spam folder, some emails have this different message:

"Why is this message in Spam? It contains content that's typically used in spam messages."

I'd guess this one means they inferred that it's spam based on the actual words in the email, whereas "It's similar to messages that were detected by our spam filters." might mean that it has a small edit distance from messages that have been explicitly marked as spam by many users (though it's definitely vague).

If that's the case, then it seems like displaying one of these two messages is pretty much all the information they could possibly give without actually saying exactly how the spam filter works.

I was thinking the same thing. It would be really interesting if it went into details about which aspects of the message were considered spam-evidence, and how strongly they weighed in the decision. That might not be a particularly useful feature, and Google might consider the specifics of its spam filter to be trade secrets, but it would sure be interesting.

Unfortunately, I think such information would be too helpful to spammers.

I guess lets you know it's Google who threw it in the spam folder not one of your filters.

Seems like that would be giving ammunition to the adversary.

I have been browsing though my spam filter:

For some message it says "It contains content that's typically used in spam messages.", e.g. a newsletter from amazon.de.

For other it says "It's similar to messages that were detected by our spam filters." e.g. the AWS newsletter or a status update from Tuenti.

I wonder what the difference is.

One message from facebook said "We could not verify this is actually from facebook.com. Be careful clicking on any links." But I am sure it was real, because it contained an actual status update. It's strange that they put into the spam box, because they could have also put it in my inbox with a warning.

wow - what is up with their blog layout design? What happend to the simple google blog style?

It is not just the blog style. Every Google product has been ruined by the UI changes. Google Groups is practically unusable now as is Gmail. As for search, pg said it best in his essay

"Google used to give me a page of the right answers, fast, with no clutter. Now the results seem inspired by the Scientologist principle that what's true is what's true for you. And the pages don't have the clean, sparse feel they used to. Google search results used to look like the output of a Unix utility. Now if I accidentally put the cursor in the wrong place, anything might happen."

In general, UI "designers" seem to be running amok at Google, with little or no management oversight. Or maybe there are too many managers pushing this stuff, I don't know. What I do know as a user is that Google is at the lowest point in its history from a usability perspective.

Yes, it's amazing that the product of so many brilliant people can be so unpleasant to use. Must be a structural problem; an organization where design is given little authority or cultural value.

This is exactly how most websites using javascript feel to me.

I really like how they reveal this information when it comes to sorting email into regular inbox and Priority Inbox. It's really cool and useful to see why Gmail put a particular email into my priority inbox.

I looked over at my spam folder, found one of the emails that was filtered due to me marking a similar message spam, noticed that it was probably a mistake on my part and fixed it. Previously, there is just too much to pick through to find those cases so I never bothered looking in my spam folder. Small, but nice!

I'm still waiting for the Gmail team to explain how Google Analytics mail regularly ends up in my spam folder.

Reply on news.ycombinator.com