Hacker News Reader

Comment by dasil003 | Hacker News Reader

I could see 20% false positives on spam for Linus equating to 0.1% of false positives across the board since I suspect the people emailing Linus are 200 times more likely to be running their own mail server than the general public.

View on HN

I agree completely. Wasn't there a thread here a just few days ago discussing Gmail consistently marking personal mail servers as spam?

EDIT: Found it. https://news.ycombinator.com/item?id=9855030

I use DKIM and have Reverse DNS + SPF and gmail likes my email just fine.

However, previous to setting all that up gmail would commonly mark my messages as spam. I haven't tested it in the last few days.

I have all of that and it took about 2 weeks of people pulling my messages out of spam for it to work smoothly.

Were they set up initially, or after you noticed problems? I wonder if prior messages routed to the spam folder that people haven't marked as non-spam count against you for a certain prior period.

I added them after I noticed problems really. Postfix and Dovecot have somewhat of a learning curve (I started up and trashed a few VM's before I got it right). I ended up using IRedMail, the defaults are pretty much Gmail ready.

I don't know about prior messages counting against you, given what I've seen it seems to makes sense. Without insider info we can only speculate.

Which is just as well, since even if you can set up a mail daemon, if you can't set up the rest of that, you have no place running a mail server.

And the moment someone accidentally clicks the SPAM button you'll find yourself with weeks of pain on a low volume mail server.

Because, as an individual, you won't qualify for their FBL service and "mysteriously" you'll have weeks of everyone saying you end up in their spam folder.

The Gmail team recently launched[0] PostMaster tools[1]. I'm not sure if everyone has access, but maybe it would help?

[0] http://gmailblog.blogspot.com/2015/07/the-mail-you-want-not-...

[1] https://postmaster.google.com/

You have to be a high volume sender with a good reputation (low spam reports) to get access to these tools.

And they don't tell you whether you meet those criteria, until after you go to the trouble of logging in and serving a DNS TXT record for ownership verification, as I just found. Granted, I didn't expect to qualify, but it would have been nice if they'd told me up front.

Yes, but at least they're using the same verification mechanism as elsewhere so if you have already added the domain to Google Webmaster for example under the same Google Account, it will be automatically verified.

The reputation requirements here are a very low bar; as far as I can tell it's effectively "don't be a botnet".

So they send you five million "I'm not a robot" recaptchas, and let you through if you only click one...

I don't have access unless I use work contacts to push and I won't do that.

Fun, but it doesn't scale. Google can only do this because they have a near-monopoly on email. What would you say if I gave you tools to whitelist yourself on my email server? You'd tell me to get my spamfilter straight, or more likely, simply ignore me.

I'm not against Gmail, just like I'm not against Outlook.com or Yahoo mail or something. It's just that providing tools only work for players in a power position (i.e. Google) who can afford to ignore small players (i.e. me), and what's more, this further strengthens their power position: the better they can detect spam so more people will start using it (the postmaster tools are there to help people prove they are good, thus helping Gmail distinguish).

I've seen otherwise intelligent people (not native English speakers but still) use mark as spam as an alternative to delete.

This was years ago but I guess that person was not alone.

Yeah. It happens and when it does, it isn't them that is negatively impacted but you :P

I had exactly the same setup, but I had to create 10 fake Gmail accounts, add the email address to the address book and flag several emails as "not spam" before it was useable. Google Mail just ignores the fact that there are private people who want to have their own mail server to be independent from Gmail.

<hat type="conspiracy">Not "ignoring", more "actively penalizing"…

I'm actually inclined to agree. Google is just so hungry, they want two things: 1. your data; 2. try to take over the Internet. Bah.

Yes, Google seems to have flipped some switch or pushed some change that takes well established domains and senders from that domain and for reasons that are not well understood written them off as spam. In theory they are getting a huge 'not spam' signal back at HQ but I agree with Linus that they screwed up big time. Stuff they should know wasn't spam and have in the past not classified as spam, now suddenly is. Algorithm update fail.

For some sender's I don't even have the "report as spam" button, even though most of the shit I get from them is ads I didn't ask for. Just because it's from a well recognized company domain.

This started hitting several of my mailing lists a while back. Not just personal servers, but several major web mail providers, most notably Yahoo and AOL. Some theories floated around that it had to do with a change to header/certificates, but I don't recall if I saw any actual confirmation of that.

Pretty ironic, then, that many suggestions in comments (on the source) are "you should just run your own mail server". Maybe two 'wrongs' do make a right..

This hit us at work really badly: we use GApps, and everything from our internal list server (which we can't replace with GMail aliases because $REASONS) was being sent to spam ... and there was literally no way to whitelist this across the company automatically.

You know how Google treats its free customers with utter contempt? I can assure you, they treat paying customers with the same contempt.

Even if said personal servers have DKIM and SPF set up? (Just thinking out loud here, I don't have data either way)

Yup, I saw a lot of people complain about this, despite having DKIM and SPF

The hilarious -- or maybe really sickening -- thing is that Gmail itself has also become a much larger source of spam than it used to be.

Those of us that are running mail servers are really not loving Google at the moment.

It's what happens when you can control such a large portion of a resource. Just imagine, Google can at any moment blacklist just about anyone. Most websites and companies they can do nothing to stop it.

It's a tad terrifying.

I can see where it would be in Google's interest for people to throw up their hands and decide running their own is just not worth it.

That is why I don't need to able to send from my server to Google. Accepting tyranny is just not worth it.

Haven't a clue what you mean. I run several mail servers and have no issues whatsoever with GMail. And I run through all the spam filtering, too, and only recall one that was inadvertently marked as spam but, iirc, the mailer was the problem.

I wrote a daemon that monitors a mailbox that users can submit spam to, for spam that makes it through the other filtering layers. The daemon finds the original message in the user's mail directory, reads the headers, tracks down the originating network using whois, and then blackholes the network and emails me a notification about it.

Some networks are whitelisted for practical purposes. Google's one of them.

If necessary, I could provide a list of the subject lines of the emails that have been reported. A YC company, Zenefits, is one offender that comes immediately to mind.

But not until Monday. I'm going camping.

While we're waiting for Monday, here are some subjects of spam emails I've received from Gmail's SMTP servers recently. Mentions of my domains replaced by example.com:

   Example.com SEO Issues
   Tough Times With Example.com? Needs Attention!!?
   Get ranked higher: Example.com
   Google optimization for Example.com
   Get ranked higher : example.com
   Website Audit Report to increase website traffic
   Example.com - audit report now available
   Give a glamorous new look to your website
   Web Design Proposal example.com
   Organic SEO Promotion For Example.com

So looks like all SEO and web-design advertising.

What a coincidence, that's Google's core business!

Back. Here's a recent selection of subject lines, extracted from spam daemon replies. The "Zenefits + Problem" ones are funny .. my dba is "No Problem", and they're not smart enough to correctly handle that I guess, which makes sense, since they're also dumb enough to spam me. I've never replied to any of their messages.

    "Re: Zenefits + Problem"
    "Higher Targeted Traffic: Associatedtechs.Com"
    "Re:re: UL,CE,ETL Split-core Current Transformer(0.333V or 1A/5A output) ,Rogowski coil,hall AC-DC transducer"
    "Poor support processes could be costing you customers"
    "Get ranked higher:"
    "Higher Targeted Traffic: Associatedtechs.Com"
    "Re: Zenefits + Problem"
    "Re: Zenefits + Problem"
    "Your Website...!"
    "Get ranked higher: associatedtechs.com"
    "Google Update for: associatedtechs.com"
    "How to increase your website traffic and generating leads??"
    "Zenefits + Problem"
    "Mobile Apps Development"
    "Digital marketing proposal- www.associatedtechs.com"

This isn't a complete list, sadly, since it turns out the daemon isn't logging ban attempts against whitelisted networks. I should fix it so that it does.

I see spam from gmail all the time, and for about the last year they wouldn't accept reports from SpamCop (that changed a couple of weeks ago).

I decided to do a quick check of my gmail; there were 151 emails in spam, and literally not one I would want in my inbox.

It seems entirely likely that Google is weighting whether something comes from a private mail server very heavily, and Linus, being who he is, gets a lot of email from private servers.

Not saying this isn't a problem, but it probably doesn't affect 99% of email users.

Just FYI, most spam does not even reach your spam folder.

What happens to it? It gets blackholed somehow?

Mail relay servers deny its acceptance.

Google can also decide it's not even worth showing in the spam folder - the spam folder is for things it's less certain of.

True. We send thousands of emails a week and all of a sudden a large portion weren't even delivered. Turns out it's all DMARC and once those policies were fixed (assigned at all) the delivery denials ended.

This helped us:

https://dmarc.postmarkapp.com

"I would want in my inbox"

Unfortunately that's not the definition of spam.

But that is his/her definition of spam. And as someone who has worked in the anti spam field, I'm ok with that.

However I have the same problems Linus has so I am not excusing gmail one bit in this.

But the people with a warped view of spam ruin it for everyone else.

So by your generous estimate, it affects more than one million users. Sounds kinda bad.

> Not saying this isn't a problem, but it probably doesn't affect 99% of email users.

So, with a conservative one billion email users, only 10 million users see this problem.

This is pedantry, but 1 billion isn't all that conservative. GMail announced ~425M users in June 2012 and announced 900M a month and a half ago.

Thanks for pointing out the real numbers. In my defence, I was thinking "conservative for the near future", so the next 5 years or so. An of course giving 9M users a bad experience isn't that great either.

If we're being pedantic, it's probably worth noting that the "99%" was a completely arbitrary assumption, and trying to run statistics based on one anecdote and an unfounded assumption is not going to produce accurate results.

True. I suppose the main point is that with a billion users, even some "small fraction" of users having problems, is going to translate to a lot of people having problems.

Honestly, if Google took support even half-seriously (or: they considered eg: users of gmail their customers, rather than just their advertisers) -- these kind of issues wouldn't be so bad.

I do think it's just a question of time before Google relegates itself to irrelevance through a strictly inferior product though.

> I decided to do a quick check of my gmail; there were 151 emails in spam, and literally not one I would want in my inbox.

I find about 2 a week that have been marked as Spam and shouldn't have been. Your experience is not everyones (nor is mine).

Oddly, I have had the opposite problem. This past week I have had to delete 4 or 5 obvious V!@g&a!!!!!-type spam emails from my priority tab of my inbox. I can't remember the last time I had that happen. Something isn't working quite right.

I wonder if they rolled out some kind of new algorithm, or major algorithm change before it was ready for prime time.

Have been seeing the same thing here as well. With spam messages coming through as normal mail.

I have also been having any messages that are replies to messages I send being diverted to my spam folder randomly. It is happening on both my personal gmail account and on a business account.

It seems lately Google has really been less focused on the core components that made them successful in the first place. I have found their search results seem to be returning more spam sites than before. My vote is to get Matt Cutts to come back and start cleaning up spam again :)

His point about marking one email in the middle of a thread is inexcusable.

I've seen that happen too. And only recently.

I've seen it happen too (from my parents no less, who I have replied to 100s of times), but not recently.

Yahoo's spam filter also does this.

True, but the spam filter could certainly learn to weight private mail servers as Ham on a per-user basis. Perhaps the learning algorithm can't generalize the feature to new unseen private make servers. However, Google certainly has the engineers who can add that functionality, they just need to properly manage them to get it done.

Yeah, its part of why I gave up on running my own mail server and signed up for Google Apps back when it was free. Its just not worth fighting the spam filters for side projects.

It was probably something along the lines of "Well consumers are what matters and they all use major services".

> I suspect the people emailing Linus are 200 times more likely to be running their own mail server than the general public.

Assuming non-gmail servers are spammers is a pretty epic fail.

What about using it as one of many criteria (maybe whether the email seems to contain gibberish being another, which patch files might be classified as).

Out of all the people in the world who regularly get mailed "gibberish" patchfiles, and not only fail to mark them as spam, but continue to interact regularly with the senders - do you not think it's reasonable to assume Gmail might notice Linus has been doing this for two or three decades?

I wondr what other forms of "gibberish" Gmail classifys as spam? GPG encrypted mail? Mail containing public ssl keys or CSRs? ANy foreign language not regulalrly hard in Bro-ville, South Bay?

Value judgements aside, that's not what I'm postulating. It's more like "unknown" mail servers (meaning any mail server with low enough volume that Gmail doesn't have an opinion about it yet).

Speaking as someone who is at least 200 times more likely than the general gmail-using public to receive mail from completely normal, mainstream Chinese email addresses... I'm still mad at gmail for just assuming mail from China must be spam. It's not spam!

(There's been improvement - for example, recently I received mail from someone I'd corresponded with in the past, and it wasn't initially marked as spam. Gmail used to be more aggressive than that, such that it would be marked as spam unless it was a direct reply to an email I had sent.)

(...for extra irony, that recent message was "I'm stuck in England and can't get home without a few thousand dollars". Her account had been hacked.)

What about the possibility that 'vanity' email servers are more likely to have something non-obvious misconfigured? Maybe, for example, they send spam rejections back to the envelope from address (generating backscatter which looks like spam), rather than rejecting spam within the SMTP session?

FWIW, I just checked my mail from the last week, and had 4 false positives, out of ~80 mails. That's WAY higher than I've traditionally had with Gmail, where I might expect that many in a year.

Yeah, whatever the problem here is, it's almost certainly very user-specific, depending on the mix of email they get, and Linus is probably somewhat atypical.

I saw his original post on G+, and of course immediately went and checked my gmail spam folder.... and... no false positives at all, 100% correctly identified spam.

The funny thing is that according to the comments, some mail from Google itself is being marked as spam.

Last year google paid me $1500 for finding a security flaw in Chrome. The mail they sent me with the ACH information was marked as spam/phishing:

http://i.imgur.com/Icfeh60.jpg

He said it mostly affected mailing list messages. His subscriptions come from a single server, so random spam classification of messages in the middle of some thread doesn't really make sense. Not as a sending-server issue anyway.

You're probably right. Mailing lists (of the discussion group variety, not marketing mail) often have problems with spam filter false positives, most commonly due to DMARC policies.

There's not really a great solution to that at the moment - either you technically violate RFCs by having your your discussion group software modify some headers, or you deal with other kinds of breakage.

Doing header rewrites is effective for reducing FPs due to DMARC, but adoption is far from universal - off the top of my head I'm not even sure if Mailman supports that at the moment.

You only have to rewrite headers if your mailing list is actually modifying the mails i.e. doing a MITM attack on the mail flow. Some mailing list admins feel very strongly about footers, subject line tags etc and then claim they "must" rewrite the From header, but I am not sure it's technically required.

Discussion groups retransmit messages, which is enough to fail authentication in a lot of cases.

Here's an example: you have an address @google.com, which has a DMARC policy of 'quarantine'. You send a message from this address to a discussion group, which in the process, resends your message from a non-google server, thus failing DMARC.

Google's DMARC policy says that if an ISP receives a message from a @google.com From address and the message fails DMARC, that ISP should place the message in the spam folder.

So it boils down to: does a list operator change the From address in distribution group mail to use a list address they own in order to pass DMARC, or do you deal with the filtering consequences of failing DMARC for many domains?

The whole point of DKIM is that messages can be relayed without breaking authentication, because it uses digital signatures instead of sending IP. So I think it wouldn't break

... IF the body is not modified, and the header signature matches, AND headers retain DMARC alignment... the reality is that retransmittal (as opposed to just relaying) almost always does one or more of these.

Here's an example from a Google email engineer's recent post to the Mailop list, which is running Mailman software.

Authentication-Results: mx.google.com; spf=neutral (google.com: 2001:41c8:51:83:feff:ff:fe00:a0b is neither permitted nor denied by best guess record for domain of mailop-bounces@mailop.org) smtp.mail=mailop-bounces@mailop.org; dkim=neutral (body hash did not verify) header.i=@google.com; dmarc=fail (p=REJECT dis=NONE) header.from=google.com

That message says the body was modified. The solution is simple: don't do that. Your original message said DKIM breaks if you simply relay mail, but it isn't correct.

I don't understand why you think that makes this acceptable.

I don't understand why you think I think that makes this acceptable.

The way you worded the sentence is classically used to indicate "well, this is an anomoly, because he is not a normal user, and is maybe even doing something absolutely abnormal, making this a trade off between the needs of the many vs the needs of the few, one he can easily bypass by doing something different inside his niche"; it took me reading a few of your later replies before I felt you were just providing an explanation for how Google's math could be flawed, as opposed to providing an explanation for why this might be an acceptable casualty.

I don't know about the spam filters but Torvalds' rants when he's onto something are awesome.

Well, as he said, some emails on the middle of threads were marked as Spam

Funny thing that happened to me, a mail from Google was marked as Spam. This was a long time ago, and it was from a mailing list, but apart from that, it was a legitimate mail.

This happens because of DMARC. It does actually make sense, in a way.

DMARC allows a domain to say "email that claims to come from my domain must be signed by me. If it doesn't, burn it with fire, no exceptions". So Gmail is only following the instructions laid out by the sending domain.

This is helping to make the email ecosystem a lot more robust by ending the problem of From forging. Ordinary users rarely realise that the From header is otherwise meaningless so phishing them can be very easy.

However it does not play well with mailman's default settings, and a lot of mailing list admins refuse to help the email ecosystem become more secure (whilst often PGP signing their own mails, doh). So DMARC creates a lot of noise in the technical community from people who have to/want to use mailman based lists.

On top of that, how many of those thousands of threads are unique individuals.

I would venture a guess that many or most of them are coming from similar people, or at very least similar domains, or even the same mail server.

I got problems this week with mailing lists routed through apache servers & for which I had filters created, etc. And this happens not only for mailing lists

yeah, things would be way better if everybody just used gmail!

Reply on news.ycombinator.com