Hacker News Reader

Comment by prlambert | Hacker News Reader

I'm one of the PMs for Gmail and hang around HN quite a bit. This is my personal take, not an official reply.

It's simply not true we have no incentive to fix this. Here are a few:

Firstly, Gmail's success is entirely predicated on the health of the global email ecosystem. Gmail does not, inherently, have any network effects (unlike FB, messengers, any other comms tool of Gmail's scale). Email itself, of course has a huge network effect, and that is because you can email anyone in the world, regardless of what email system they use. It's because email is open. If we lose an open, healthy ecosystem with many providers, we'll destroy the base we stand on.

Secondly, we care deeply about having positive relationships with developers and all our users. I can tell you it definitely makes me sad to see articles like this. There are going to be false positives, we will make mistakes, but we certainly care a lot about fixing issues like this when we hear about them.

I agree Postmaster tools has been underinvested in and we could do much better there.

View on HN

I appreciate the response. From someone who has operated a mail server since 2002, it comes across that Gmail does not care about cooperating with small but legitimate mail servers.

SPF, DKIM, reverse DNS, no blacklists, no open relay, longtime ownership of IPs, etc etc. Using various mail testers returns a 10/10 deliverability score.

And yet, messages sent to Gmail always go into the spam folder, or are never delivered at all. These are everyday regular messages, I have never used mailing lists or sent bulk automated messages.

The issue is, there is no recourse, no fix, no acknowledgement of the problem with false positives. There is no tool available to me to understand or correct the "problem". Hint: this comes across as Gmail not giving a shit.

Gmail has a responsibility to be more accountable, even if these problems are unintentional, because Gmail is such an enormous node in a federated network.

> If we lose an open, healthy ecosystem with many providers, we'll destroy the base we stand on.

Correct. Gmail is contributing to the erosion of email reliability. Please course correct.

I've had gmail send the following to the spam folder:

- Legitimate class action notices related to Amazon purchases.

- Email coming from addresses to which I had already sent email. (!)

- Email from my landlord.

- Email coming from Google itself.

Based on the contents of my spam folder, which I have to check fairly often because of the extreme overaggressiveness, I would be vastly better off if nothing ever got filtered at all. [1]

>> There are going to be false positives, we will make mistakes, but we certainly care a lot about fixing issues like this when we hear about them.

This doesn't sound honest, or at least not complete. People have been complaining about this for years. I have personally been complaining about this for years. The loss of obviously legitimate email is completely outrageous.

It doesn't look intentional (look at that fourth category!), but it certainly doesn't look like anyone is trying to address the problem.

[1] Yes, if spam filtering was disabled, more spam might get sent.

>Legitimate class action notices related to Amazon purchases.

You'd think Google had an incentive to get that delivered.

> - Email coming from Google itself.

Yep. I missed an invitation to a Google-hosted event at a conference I attended because the email (from an @google.com address, no less) got caught in Gmail's spam filter.

This is a problem with ML approaches, right? Instead of water boiling at "100C" it boils at "99.98C +- 0.04C". Normally this is ok, but sometimes it isn't!

If something happens a trillion times a day with .00001 error rate, then 50 MILLION things went wrong, affecting, say 50,000 people.

50 of them blast twitter and it seems the world is collapsing.

The world is resilient.

I don't think an average of 1,000 mistakes per person-affected-by-mistakes is realistic here. I'd bet it's closer to 1.

If there are 50,000 people and 50,000 people experience problems, that's bad.

If there are 5 million people and 50,000 experience problems, that's fine?

Isaac Asimov's comments about world population increase involved something about this; the more people there are, the more each individual is dehumanised and rendered irrelevant (my paraphrasing).

No I don't think it's fine at all. I think Google, twatter, Facebook, et al don't care because 50 people and who they represent don't matter to them compared to the money they make.

When all rounded up it isn't even a single penny on the balance sheet. The owners of these businesses literally never even know from the their only view into the companies.

I have no idea why I'm being downvoted on this. Hackers can't do math or what?

I think your position is a little unrealistic. 50 people experiencing problems out of what, a billion? is pretty good. Do you think that if those billion people were served by 20 million small business e-mail providers, that none of those 20 million e-mail providers would ever make a mistake and affect their 50 customers?

I said 50,000 were affected. Only 50 twatty tatted about it.

OK, 20,000 e-mail providers with 50,000 users each. Does that change the point?

Sometimes error rates, even nominally low ones, are bad. Would you like to have your IV changed by a nurse with 0.001 error rate?

I imagine most humans have error rates worse than that. And what does 'error rate' even mean in that context? A small delay or a catastrophic failure ending in death and destruction?

More importantly would you like to have your IV changed by a nurse that makes up her own standards for health care?

Google does what's best for Google - how could there be any discussion of that fact in 2019.

I would think that would be quite a good error rate. Especially when considering people in the hospital are often not in good health, possibly making their veins more difficult to find.

That is assuming by error you mean missing the vein. If error is defined as a fatal complication, then 1/1000 is terrifying.

Denver is just a small minority of the US and we just can't please everyone.

(afaik in denver water boils at 95C)

In Denver water boils at 203F

Yup, I've had the same thing. Just kinda amusing and ironic since I didn't happen to care about that event but it makes one nervous about relying on spam filtering.

On the other hand, once you train it a bit, it is mostly remarkable good. For me, switching from fastmail.fm (which was pretty good itself) to Gmail gave me a big improvement in spam control.

Super curious about this response saying on gmail you have more control, because from where I am the “mark as spam” button does nothing but move things to the spam folder. In theory it should learn from that but when someone used my email address to sign up for AT&T no amount of marking things as spam will stop their emails landing in my inbox.

As in signed up for AT&T service? If so it's because it's not spam - it's misdirected mail, but there are tens of thousands of other Gmail users who think that messages almost identical to those are things they absolutely want to receive.

If there is no business relationship between AT&T and that user, the U.S. Can Spam Act defines it as spam.

Shame on AT&T for not validating their customer's email address.

I too get the same type of spam from AT&T.

My point is that absent information that Google simply does not have no matter how creepy they get, there's literally no way they can identify such messages as spam - exactly the opposite in fact because probably 99.999% of such messages that they process are explicitly not spam.

The only way Google would have to identify that this message was not for you would be to get the subscriber information from AT&T and cross-reference it with name and address information they had for you - and even then most of the time they'd probably be wrong (e.g. if the email is coming to you but the account is actually in a family member's name).

You know what's really funny? That even with that overly agressive spam filter, once in a while (once a quarter maybe), it somehow manages to miss obvious BuY@@NiGERiAn@@Vi@gRa-Cia1is type of emails... which, by the way, my morally outdated spamassasin marks as spam.

"morally outdated" ?

I would make an educated guess that "arpa" is from EX-USSR and their native language is Russian. It's direct translation of idiom "морально устаревший" which literally means something is:

* Available for decades.

* Far from being top notch technology.

* Sometimes of course it's literally mean outdated. Like if you run older CentOS or Debian with decade-old packages.

So it's doesn't mean SpamAssassin is bad, but it's very far from state-of-the-art ML technologies that Google might have.

Bulgarian has that too: "Морално остарял"

Before I switched to fastmail, gmail would not believe me when I marked an email as not-spam. They would still get flagged. All I wanted was updates from one band.

That didn’t stop me from repeatedly getting junk from every other record label my email was sold to. It was an endless procession of shit I never subscribed to.

"Inboxing" with Gmail & Outlook is a baroque, hellish process for mailservers.

When either provider decides your small email server is sending spam (eg: sending an email with an attachment, or any kind of form email like a daily report) you won't get through to user inboxes, and instead you'll be routed to spam, or for Outlook.com hosted addresses they will accept mail from your server and send it to /dev/null. Gmail's process is bad, but Microsoft has decided to accept emails and throw them away (which is ridiculous).

At my workplace we have two email addresses, one from google mail (gmail for business) and one from MS Exchange.

I never had a problem with gmail for business regarding spam. I regularly receive mails from smaller businesses (some of them hosting their own mail server) and never had a complaint from anyone yet. Since i can also be contacted via phone i'd know.

On the other hand, MS Exchange constantly delivers obvious spam mails and (quite seldomly, but still) swallows legitimate mail.

Anectodal, i know. And disclaimer: the behavior depicted in the article is as bad as it gets, if everything is as described.

As far as I know, Gmail also does this sort of black hole spam filtering - the stuff that you see in "Spam" isn't everything.

Your comment motivated me to check my Gmail spam folder and 10% (3 out of 30) were false positives, two of which were pretty important. :(

Me too. I found an invitation for an on-site job interview I would have 100% missed if they had not called me as well.

Likewise I found they filtered an e-mail from my bank about my car loan.

I've recently started checking quite regularly my spam folder as I've noticed more and more legit emails ending up there. One of them being support emails from TradeMe (one of New Zealand's biggest sites), keep getting put into spam, even after multiple "mark as not spam", along with some other kinda important emails from TradeMe. I've had to put in manual filters to force an email from TM to skip the spam.

And yet, in my own none Gmail hosted email (fastmail), I currently have 2/55 false positive spam emails. I very rarely ever check it. To note I usually get one actual spam (non newsletter blog spam) to my actual inbox, a month.

TradeMe, like eBay and Banks is a popular target of phishing emails.

I recently missed an email from the UK government regarding a passport application, it went to Spam. Talk about missing the mark!

A spam filter can't whitelist government email. My personal SpamAssassin filters out spam from government servers all the time. The latest was from somewhere in Quebec.

You'd think the various governments would put more effort into computer security. They appear not to care, though.

Me too. 2 "spam" emails in Gmail's trash folder, none of which were spam (one was a newsletter from Mastodon (software)).

Edit: Gah. Now I checked my work Gmail spam folder. There was an email from one of my users there. (and nothing else)

Now I'm considering migrating away from Gmail, at least for work related things.

Ditto, Multiple missed emails from factory owner in fuzhou I'm on site doing business with and actively emailing back and forth with daily. Despite the back and forth communication some of the direct messages were in the spam folder. This could have caused me some major issues.

About accountability, this gmail thing vaguely reminds me about "The Drop" situation on the mobile phone network https://www.youtube.com/watch?v=pCOCKS5AJI8

And all parents with kids in SF public schools should note that all emails from noreply@sfusd.edu (bulletins, etc.) are marked as spam.

> - Email coming from Google itself.

If email coming from google itself got special treatment there would be masses with pitchforks complaining about that.

You've made a lot of good points, but I don't think that's one of them.

I don't think that has anything to do with Google needing special treatment. It shows that the company who made the rules and has every privilege to follow them isn't able to.

I think that's a fair interpretation and a good point.

Valid point. They've become so large and synonymous with email that many people I encounter are actually unaware that there is other email besides "Gmail."

It feels like Google no longer has any incentive to follow the rules, and they feel that they are going to be the ones to make the new rules. The rest of us end up having to implement workarounds.

> You've made a lot of good points, but I don't think that's one of them.

I disagree. I think in the absence of that point, it would have been hard to say this:

> It doesn't look intentional (look at that fourth category!)

But also, I think special treatment for trusted actors is a completely appropriate way to handle email delivery, and I also think it's appropriate for gmail to trust themselves to be sending legitimate mail. Blocking their own email makes them look totally incompetent. They absolutely should whitelist themselves. And they should have a way for you to be whitelisted too, if you want to send email.

If they whitelisted the major players in the space, I'd worry that the problem would get _much_ worse for everyone else.

>> I would be vastly better off if nothing ever got filtered at all.

There are layers of filtering beyond what appears in your spam folder, layers that block obvious spam long before it gets anywhere near your account. If every email ever sent to your address wound up in your spam folder you'd beg for filtering.

I'm prepared to believe this. But it's not a defense of gmail's policies -- this suggests that in fact nothing would be lost if gmail eliminated the spam folder and delivered everything that would have gone there to your inbox instead. So why are they doing this?

Before switching to shared hosting from my VPS (one reason being that i didn't want to bother with email maintenance and didn't want to have a separate service just for email), i had my mail with (badly configured) Spamassassin that wouldn't delete mail, just add a "SPAM" prefix in the topic. My mail isn't exactly commonly known, but still is one i have for years and was made public thanks to me releasing an Android app once (after which, spam increased dramatically).

Even after running it for years, Spamassassin never marked a legitimate mail as spam, so i'm pretty sure that if i wasn't too lazy to configure it to move it to a spam folder, it'd work fine. Stuff did pass through it (at a ratio of one every four or so) but i was fine with deleting those.

What i'm trying to say is that from personal experience, i'd be fine with a spam filter that errs on the side of not marking stuff for spam and me deleting whatever goes through manually. Having to see a bit of spam mail is small cost for losing mail i'm actually interested in.

I'm using my e-mail pretty much everywhere (me@vbezhenar.com you have it in clear text, every bot will crawl it now, not for the first time, though). I'm too lazy to setup spamassasin yet, so I'm getting a notification for every spam message I got. The only thing that I'm trying to do is to click "unsubscribe" even from obviously spam mails (may be it'll make more harm than good, not sure). So I'm getting around 20-30 spam e-mails per day. I don't think that it's THAT bad. I'm spending may be a minute every day to delete it. And I'm sure that with absolutely minimal spam filtering I would achieve almost perfect filtering.

I had a communication from a Google recruiter go to my Gmail spam folder.

But it still beats no filtering.

I've also had email from a Google recruiter go to the spam box...

I think the SPAM folder is a separate issue (although important)

The article discusses mail not being accepted by google/gmail in the first place.

Fair comment. But that issue is pretty simple: there is no good reason for them to fail to deliver email under any circumstances. (This has happened to me too -- someone tried to email my gmail account and gmail completely refused to deliver it. It was pretty embarrassing.)

Messages they think you won't want to receive are what the spam folder is for.

"fail to deliver email under any circumstances."

The amount of spam that would be delivered if they didn't discriminate AT ALL is enormous.

They have to read all this feedback and discriminate better.

> there is no good reason for them to fail to deliver email under any circumstances

Yes there is. They don't want to carry traffic from anybody from the major email blacklists. If a mail server is on a real, very transparently-managed blacklist, no large provider should be accepting their smtp traffic.

Yet all the people here trying to administrate servers from residential and VPS blocks of IPs are telling you they're caught up in this list you're lauding as all-knowing and safe.......

I've also had email from Google recruiters (@google.com email addresses) go to spam. I considered this a high level indicator that Google spam filtering is incompetent, not malicious - if it was malicious, they would at least be able to get their corporate emails through.

- Email coming from addresses to which I had already sent email. (!)

That is understandable. It is hard to validate if an email is authentic. SMTP has no authentication built in. Gmail can't just blindly accept all emails from addresses that you have already sent an email to.

EDIT:

Look at this example: https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol#...

Anyone can connect to relay.example.com and pretend to be bob@example.com.

Huh? If it's in the Spam folder, they do accept it.

I meant accept as not spam.

Not if you use SPF/DKIM/DMARC they can't, that's the whole point of those various additions.

All those "I hacked your email and send you a message from you account" I don't get, because I have a DMARC policy that says if you don't pass SPF/DKIM then you get rejected. So try as the spammer might to connect to my mailserver and pretend to be me, they can't, because my mailserver sees they're not authenticated, and the mailserver they're sending from isn't in my SPF records, isn't signing the message with my DKIM key and therefore it gets rejected at the SMTP level.

That’s what SPF[1] is for, and optionally DMARC[2].

[1] https://en.wikipedia.org/wiki/Sender_Policy_Framework

[2] https://en.wikipedia.org/wiki/DMARC

I just cleaned out a little over 100 emails in my Gmail spam filter yesterday. About 80% of what was in there were emails from YouTube giving me notifications of new videos people have uploaded that I am subscribed to. These emails never used to go to spam, but slowly over time more and more of them would end up in spam. It's at the point now where almost all of them from YouTube go to the spam folder.

It doesn't make any sense since they are emails from Google, they are emails I even have a filter applied to so that a label is applied to them. Yes I can adjust the filter and choose "never send to spam" but the messages will still show a warning on them saying "This message was not sent to spam because of a filter you have applied".

Sure false positives makes sense, but I don't get how the majority of what is in my spam folder would be emails sent by Google.

> It's at the point now where almost all of them from YouTube go to the spam folder. It doesn't make any sense since they are emails from Google

It makes a lot of sense... people use the spam button as a lazy man's unsubscribe. Youtube adding the bell button, making mail opt in is probably a response to that.

I actually give Google a lot of cred for not simply white-listing its own domains. Though spammers would probably find ways to abuse it and make them look bad anyway.

> people use the spam button as a lazy man's unsubscribe

This is the small mail server crux right here. If you’re a small mail server and a few of your emails have been spam binned instead of unsubscribed, it would likely lead to your whole server getting shit canned.

> - Email coming from Google itself.

To be fair, sometimes Google does send spam.

Normally I wouldn't +1 a message because it doesn't really seem to add much to the message. But in this case I will (even though it might earn me ire from some). If I could highlight the parent in bold I would. I've run mail servers since the mid 90's and the experience above exactly mirrors my experience. And my takeaway is the same, GMail just doesn't seem to care. I understand that GMail can't give away too much info on why a particular email ghosted (opsec and all that) but GMail doesn't need to do a better job of explaining to postmasters what good actors can do to avoid being ghosted, and more importantly have that reflect actual real life experience.

Add me to the "I've run mail servers since the mid 90's and the experience above exactly mirrors my experience." list.

I think all of you should start some kind of a movement so that this could be heared better at GMail's end.

That's a great idea. I've run my own mail servers since 1999. I'd happily join a union of independent mail operators with the purpose of lobbying Google, Microsoft, and Yahoo to treat us as first-class citizens.

Start a mailing list? Or is there an existing one for this topic?

I think you should have a web page where you list your issues, show how you've done all the normal things to be considered a well-behaved mail domain. And then perhaps report your experiences with iinteracting with gmail support on the matter.

Or even better: so that people stop using Gmail.

Even if it isn't deliberate, even if gmail give a fuck, as can be seen from the first (and so far only) response from anyone at Google, they're so far in the Google "We're better than everyone else and always get it right because we're geniuses" RDF that it literally doesn't matter. They can't believe that they're the problem.

Me too, but early 2000s.

Early 2000's for me, and yes, similar experience. What was really interesting to me is that my home domains had no problem emailing my work and side bit gmail based mail service a few weeks ago. Then, suddenly, this stopped working.

I changed jobs as well, and now work is using a MSFT based hosted mail service, and I am getting delay messages.

Seriously, GOOG, MSFT, and others broke mail. This is not an improvement.

I've not looked into speaking with MSFT mail folks about their breakage yet. With GOOG, you have really no mechanism of reaching out to someone there and getting attention for the problem they are causing.

This is the much bigger problem with GOOG actually, in case any googley people are reading this. They just don't get customer service. At all. It is near impossible to be able to report a real problem across the spectrum of their services. Unless you are one of their bigger customers, you don't have access to even a telephone support number. Their online help is a crapshoot, with you getting useful information less than 50% of the time.

So where I am now is with locked down, long time existing domain mail servers, which send maybe 5-10 outbound messages per month, that suddenly and inexplicably, have a bad reputation. Well, no they don't have a bad reputation, they can send email just fine to other services.

> GMail doesn't

what?

Me too, since 1997. Same IP for over 12 years.

Thing is, most of my messages get delivered. Then suddenly they don't.

Google "giving a shit" means responding to hostmasters about delivery problems, and they just don't.

Please note this, Gmail PM.

After reading this thread I checked my spam and it seems google has gotten way more aggressive in the past ~ 6 months when it comes to spam. They are mass mailers and advertisements but they are moving items that I want -- legitimate subscriptions and people I do business with are ending up in spam. Never experienced such a high failure rate before and I have had gmail for about 15 years.

I will chime and validate this experience. You will not fully understand or realize the hostility google has towards open email unless you have working experience with email protocols. This is by design.

It is my belief that this is intentional and I would love to be corrected if it’s not.

Reminds me of this email from Linus Torvalds about a Gmail bug that was affecting git:

https://public-inbox.org/git/CAHk-=whP1stFZNAaJiMi5eZ9rj0MRt...

I have similar experience and of course if you send exactly the same message but from gmail account, then it's not a spam for gmail anymore..

Just to be clear, your comments are refering to problems with Gmail's Spam filtering. Isn't that a different issue than what the OP is saying--that his mailserver is being blocked? In other words, emails from his mail server wouldn't even make it to Spam folder.

Gmail spam filtering and it's mail server are two sides of the same coin. The mail server itself is blocking believed spam messages as a first layer, then the spam filter acts a second layer that marks messages as spam. I guess they don't want to save messages that they are completely sure are spam, the problem seems to be their false positive rate or simply their approach against small operators.

This is totally my experience as well. It got so bad that I gave up operating my email server.

Anyway, Gmail is getting some heat in this thread and rightfully so. We should however not forget that Microsoft and Yahoo are just as bad if not worse in this respect.

I´ve unfortunately searched for some customer-friendly e-mail provider and people want Google.

I had set up some commercial accounts on Zoho (my own e-mail is on Zoho) but they asked me to migrate to Google, even though it was more expensive.

Just checked my junk folder.

9 out of 10 threads marked as spam were false negatives.

That's a 90% failure rate.

I have everything set up correct and gmail even accepts 90% of my emails but every now and then it randomly marks one of my emails as spam. A gmail user emailed me first and I replied with a text email no links and that gets marked as spam.

> There is no tool available to me to understand

I _think_ this is because G doesn't want people gaming the system.

> or correct the "problem".

I regularly fish mail from my spam bin. From mailing lists, and other important stuff -- and indeed Google's own mail!

One thing you can do is to get people to add you to their contacts list.

It's a hard problem. There is one "solution" you probably do not want. Have Google (and other companies) give their imprimatur to certain mail senders.

First of all, if you check this site:

https://toolbox.googleapps.com/apps/checkmx/check?domain=tab...

It is not to be set up correctly.

Now in your case, you failed to mention DMARC records. ALL of the big cloud email services have required that for years. Your SMTP server checklist is straight out of 2005.

This entire thread reminds me of the guy who pulled out of a desk drawer an Analog Startac cell phone and blamed AT&T for it not working. Email has moved on. It not Google's fault that people are configuring SMTP servers like it 2005.

What does this mean?

> error SPF must allow Google servers to send mail on behalf of your domain.

So you must let Google spoof your domain? That seems crazy.

ProtonMail has the same error:

https://toolbox.googleapps.com/apps/checkmx/check?domain=pro...

He's wrong about what this tool provides – namely information for people who want to use GSuite for their email. In that context it makes sense that you have to allow Google "spoofing" your domain, of course. But it has nothing to do with what is being discussed here.

Ah. Thanks. It makes sense now. I didn't read carefully enough.

This tool checks that you have set up your domain correctly for use with Google Apps.

For domains that are not using Google Apps, its results are meaningless.

It turns out DMARC isn't actually required. Having it might help your acceptance score, but gmail and outlook don't actually require it.

I know because I haven't set up DMARC for my email domain, and my email still gets accepted by everyone.

I believe it helps that I accept on TLS with a valid certificate, and I used Google's postmaster site verification.

I sent mail from my personal gmail to my company email (different user) for expenses

Since money was involved, I paid attention and followed up

Our finance person found many emails in Spam that were important and should not be there

On the other hand, my personal Spam folder is certainly full of crap I never want to see. But now I don’t trust the system, so I have to scan them anyway

Datapoint: Ive run a small exchange server with fewer than 500 users for 5 years and have never had an issue delivering to Gmail except a couple times when we had legit issues on our end.

Have you looked into WildDuck.email? I find it very nicely made for people who wants to run their own email. Still in Beta. FYI, I'm just a js developer not related to them.

I also have a 15+ years old set of mail servers and I host mail domains for 10ish friends and family. In fact, I once wrote how to set this up (http://flurdy.com/docs/postfix)

I no longer use Gmail myself but half of my users relay some aliases to their main Gmail account. No problems with that, except my servers continuously get rate limited by Google:

    Our system has detected an unusual rate of 421-4.7.0 unsolicited mail originating from your IP address. To protect our 421-4.7.0 users from spam, mail sent from your IP address has been temporarily 421-4.7.0 rate limited. Please visit 421-4.7.0  https://support.google.com/mail/?p=UnsolicitedRateLimitError to 421 4.7.0 review our Bulk Email Senders Guidelines.

I have the normal anti-spam features set up (SPF, DKIM, DMARC, Postgrey, Spamassassin, etc) so most spam gets rejected but not all. But several of my users have very common English names as their aliases that are already guessed and added to many spam databases.

This leads to family phoning me all the time to say my server is broken when it is just Google throttling everyone's emails. I frequently have to check Postfix queues and clear some obvious spam or just pointless Facebook update emails which seems to be the majority... (Yes your email admin can read your email...)

The only way to improve is to constantly remind my family to not sign up to all crap, use not so common aliases, and try to keep tightening my anti-spam configurations. But we really are not talking about a lot of emails. Probably less than a hundred per day spread across 5-ish end accounts of which 95% is probably legit. Yet Google is treating me as some totally open relay. (╯°□°)╯︵ ┻━┻

I do understand Google's challenge with handling probably the largest amount of email traffic in the world, and then the most spam in the world, and that it is a continuously moving target.

But it does feel like they are treating nearly all minor relays as spam relays. We can not block 100% of spam before we relay onwards to Gmail as that would mean too many false positives emails get blocked, but most try to block as much as possible.

In my case, my servers probably block 99% of spam, but some will get relayed, and most of those ends up being handled by Gmail's even better Bayes scoring and filtered to end users' spam folders.

I don't know what the automatic threshold limit is to be on their naughty list, but it must be very low, as in double digits per day.

A few years ago, I was dealing with a few small business sites which were self-hosted and always had deliverability problems with the large mail services (mostly, back in the day, AOL and Yahoo).

Large providers tended to have a world view dividing all senders into two categories: 1) Bulk senders who are clearly mailing the same spam to a list of a billion addresses 2) Non-commercial individual hosts which should be sending five messages a day or less in total.

It felt like there was a huge missing third category for transactional emailers nobody wanted to acknowledge. They are probably difficult to score fairly. A hundred "Order details" emails are going to have the same level of randomness/templatedness as the old Viagra spam which had a random block of Project Gutenberg text pasted at the end to trip up filter math. You're not going to have a clear history of "this address bounced twice, let's stop sending newsletters" when most of your messages are to first time customers or once-every-few-years return ones. A lot of the messages will look generic because they use default shopping cart templates.

To the extent they provided sender guidance, it was focused around use case 1) -- sign up for feedback loops and deal with greylisting (because people really love waiting 18 hours for an acknowledgement)

> It's simply not true we have no incentive to fix this

I just happen to have set up an email server and encountered the same problems with Google as described in the article. I own the IP since quite some time, it is not on any black list, reverse DNS is set up etc. but Google rejects email as spam.

And this even happens when the gmail account has added the sender in his address book and has send the first email to which I replied - thus there is a message id that should already be known on Gmail's side.

Use your AI to put email into the spam folder. Refusing it outright is a case for the European Commission which hopefully will slap you another few billions of fine onto the wrist until you remember to play nicely with the other kids.

There is no excuse to refuse SPF, DKIM, reverse DNS, proper MX, no blacklist, sender in recipient address book and reply-to msgId email.

> Use your AI to put email into the spam folder. Refusing it outright is a case for the European Commission which hopefully will slap you another few billions of fine onto the wrist until you remember to play nicely with the other kids.

I prefer getting a bounce than ending in the spam folder by far. Lots of relatives on Gmail never look into the spam folder so I never know if they get to see my email or not and I end up having to ping them on eg. WhatsApp. At least with a bounce, I know they did not get my email.

But if it ends up in the spam folder there's at least a chance that it'll be trained out of the spam folder by people marking it as "not spam", and it provides a workaround for the people who really need it to work _now_.

I believe that the majority of the users are trained to ignore g-mail spam since it's been working good enough for most people. Unless we condition the customer to always view the spam folder it's asking them to do something on top of already checking their e-mail.

My mother would just simply ignore the spam and often times spam catches majority of the phishing e-mails too. So it's a double edge sword educating and conditioning the users to review the spam folder. Why is it the end user's job to determine what is spam and what is legitimate ?

> Why is it the end user's job to determine what is spam and what is legitimate?

Who else could determine that? I could hire you to send me emails about Viagra sales. They would not be unsolicited, because I specifically asked you to. They wouldn't be commercials, either, so they wouldn't be Spam. An automatic filter can't determine if it's spam or not, it can only take an educated guess.

Then they should do both: bounce it and also show it in a folder called 'bounced'.

Great idea. Since many of us rarely check Spam.

I wonder if the correct solution is "send an informative pseudobounced-as-spam message to sender AND put it in the spam folder", for borderline spam (which passes the other checks). Downside is it provides a deliverability oracle for spammers, but since gmail accounts are free, that's not too hard to establish anyway.

But you can mark it as "not spam" (and hope that it does something, I guess) while you (as the recipient) have no options whatsoever with a hard rejection.

It might obviate the excuse mentioned a few comments above, "I've never had a problem with Gmail, it must be on your end."

If the message is ending up in your spam folder, it's harder to shift that blame off of Gmail's shoulders.

That's as a sender (even that reasoning is questionable). As a receiver, would you rather never received an important email from some org instead of finding it in your spam folder?

I'm genuinely sorry to hear that. Our analysts are looking into the issue described in the article, hopefully we find something we can fix that will resolve this.

Since you profile claims you are from Google, you should know by helping here you really doing disservice to everyone else having same problems. Hacker News is not a “gmail customer support hotline center”. You have billions of dollars to setup a system or heck the whole state of the art department to help people with their gmail problem. So you will help two people on HN bitching at gmail.. what about all those that never heard of HN? Sorry friend you doing this only because at some degree google and gmail obviously does not want to look bad to tech society. /rant

That all depends entirely on whether the "fix" is white-listing the two responders somehow, or flagging the larger issue to the correct team and figuring something more general out.

Not sure why you are being downvoted, you have a point. There is no excuse for the way Google is operating now - poor customer support, the only recourse twits and HN posts.

I think it also hurts Google themselves. There is a reason they have trouble competing with AWS. Why should developers trust Google that there will be a person on the other side helping them out when things go wrong? And they always do, at worst possible time. Amazon, for all its problems, has excellent customer service.

So your parent's attempt to fix this is a PR stunt at best. If they (/Google) care about this, they should fix the problems in the process:

- define, hardcode and publish rules that will lead to successfull e-mail delivery (SPF, DKIM, history,...)

- establish a gmail technical customer support service

And they should stop with preferential treatment of those who shame them publicly. Or do they want all of us to start doing the same?

I don't think you can have an open, healthy email ecosystem, when the parties involved (the people running and managing email infrstructure) can't communicate with each other and get help.

> Or do they want all of us to start doing the same?

An increase in public shamings probably means that public shamings become less news- or interest-worthy. So an increase in shamings might not have as much effect as you might expect.

That’s true, but I don’t think we're anywhere near there yet. Especially when it comes to the mainstream — most people outside of tech don’t have any idea this is a problem.

(FWIW, I don’t seem to have this problem in a significant way, but that might be because I have DMARC set up.)

hmmmm, so do I and still my emails are unsolicited.

> they should stop with preferential treatment of those who shame them publicly

That's the only way companies care in the 21st century, 3 weeks on an email thread, one tweet and suddenly everyone cares.

Their customer support is just fine. Only trouble is, we're not the customer, we're the product. (And no, I'm not interested in hearing about how that's a hackneyed worn-out cliché, or whatever, given that it's a true statement. Downvote and move on.)

A big part of the problem with Google taking everything over is that only in very limited situations -- the service formerly known as YouTube Red, for instance -- are we given the ability to actually conduct business with them as a paying customer. Conventionally, Google users don't even rise to the status of sharecroppers, since we're the "crops" being sold to advertisers. They expect us to depend on them for the everyday conduct of our personal lives and careers, yet the only way to appeal for help is to start a shitstorm on Twitter or HN and hope somebody notices.

> I'm not interested in hearing about how that's a hackneyed worn-out cliché

Well, it's a hackneyed worn-out cliché. Without the users there's no viable product. That's what makes them the customers and that's why they need adequate customer support. The fact that you pay for the product with ad impressions rather than dollars doesn't change anything about that dynamic.

Well, it's a hackneyed worn-out cliché. Without the users there's no viable product. That's what makes them the customers

No, that's what makes them "users." The customers are the advertisers who actually pay money to Google. You can rest assured that they don't have to post a cri de coeur on social media to get a response from Google when something goes wrong.

IMO, a company that does its level best to act like vital public infrastructure needs to be held to standards appropriate to vital public infrastructure. If that's a controversial point of view, then so be it.

> No, that's what makes them "users." The customers are the advertisers who actually pay money to Google.

I understand your idea. But in my opinion this is a useless and unconventional definition of the word 'customer'. The customer is the one who receives a service in exchange for compensation. In this case the service is gmail and the compensation is ad impressions. The fact that Google can sell those ad impressions is secondary to that dynamic. If there were no users willing to provide that compensation in exchange for access to gmail, then there'd be no product, and there'd be no secondary market for selling those advertisements. Meanwhile even with no advertisers there would still be a product, and there would still be other avenues for monetizing it.

> You can rest assured that they don't have to post a cri de coeur on social media to get a response from Google when something goes wrong.

Are you sure about that? A quick google for "adwords support experiences" gives quite a number of telling stories to the contrary. Besides, there are several orders of magnitude more users than there are advertisers, so obviously it's going to be much easier for users' issues to get lost in the noise. And obviously no advertising agencies are going to be getting any sympathy by complaining on Twitter.

> - define, hardcode and publish rules that will lead to successfull e-mail delivery (SPF, DKIM, history,...)

This has existed for years:

https://toolbox.googleapps.com/apps/checkmx/check?domain=tab...

If your site passes and is not on a blacklist, your email gets delivered.

That says you must permit google to send email on your behalf in your SPF record. How is that relevant at all for sending email to Google?

This is not doing what you think it does. And it's the second comment where you are spreading your misinformation I've seen in this thread now. :( This tool is for people who want to send their emails via GSuite and their own domain. Not a lot of people in this thread want that, and it certainly has nothing to do at all with what is being discussed in the comment threads where you replied to.

In his first post (above) he said that he is from Google but he is not acting in any official capacity here.

It is not Google support, it is just an employee giving some insight on his own time.

What is wrong with that?

I agree with your message except the motivation part - you can't question that particular person's motives. It may very well be they care just like many other people at Google do, even though the management doesn't.

Let’s say conforming to usual behaviors? That is simply what the article is asking for.

I struggle to see how your analysts could do this without looking into specific instances of the problem. Perhaps you intend to contact the author of the article to ask for this. That might work in this case, but in the general case there is absolutely no way for people to report this problem to Google!

Yes, I'm yet another person with the same problem as the author of the article - running my own mail server for years, only I send mail from it, very low volume, everything set up properly, etc. The difference is that my emails go to spam, rather than being rejected by the SMTP servers, which to me is even worse, since I never know whether an email I sent to GMail has been delivered or not. Since GSuite is so popular now, I never know this for any unfamiliar domain, unless I do an MX lookup.

You say all the right words, but Google's actions on this issue (or lack thereof) speak much louder. It's very difficult to believe that the situation will improve and many comments here reflect this skepticism.

It's not just small fries getting hit by this. Google regularly refuses to deliver mail from backerkit.com to my account, by far the most popular Kickstarter fulfillment support service. I frequently have to contact Kickstarters and ask them to manually send me mail because Google is refusing to accept it, which is annoying and wastes their time.

I wish Google had a way you could tell it "messages from these people/domains are never spam (or I'll deal with it myself)".

Google Apps (or whatever it's called now after the last dozen name and service changes since "Postini") does offer a whitelist function, but as far as I can tell it's there only to placate users. Whitelisting domains or individual email addresses globally, or per user seems to have no bearing at all on whether they will show up in your inbox... Postini used to work flawlessly, it's hard to imagine how they could have screwed it up so badly.

It’s AI above all else. It seems instead of a rule that says all emails from these people are ok, they’re too reliant on ai filtering based on the content. They’re probably trying to prevent spoofing.

I use gmail and they know who is sending. I got an email today with a gmail flag saying “this user sent from a different email address previously”

In gmail, you can create a filter to skip the spam folder. I regularly receive emails where, at the top it says "this would have gone to spam, but you told us not to send it there".

I doubt this affects things when messages are rejected at the protocol level, but if your problem is emails showing up in the spam folder, this might fix it.

One of the problems is the pure mechanics of how SMTP works: anyone can send an email with any "from" address, and filtering on people/domains in the "from" field is basically pointless if you want to catch spam.

What Google is likely doing is checking the domain of the originating IP in the SMTP "envelope", but that also gets tricky with outsourced email services or internal IPs.

I do not use Gmail, so these are just wild guesses, but I do run my own mail server and frequently get my email not showing up for people.

Problem with SMTP bounces is that it may take a week for the final bounce to show up in my inbox (because again, that's how SMTP protocol is designed, to expect nodes to be down and retry a number of times).

> anyone can send an email with any "from" address, and filtering on people/domains in the "from" field is basically pointless if you want to catch spam.

Isn’t this the exact thing that DKIM is designed to fix?

This was a response to a complaint about not receiving emails from some "people/domains" even when whitelisted, and I highlighted how this can be hard for a service to detect reliably.

So yes, DKIM will help a receiving server know for sure, but a receiving server still needs to accept emails from non-DKIM-enabled servers, and perhaps that's why whitelisting didn't work for the parent.

Bounce handling depends on the error code, 5xx is usually a permanent error so the bounce is immediate. Mailbox doesn't exist etc. Its not going to exist later either. IIRC 4xx is effectively try again later because something is wrong right now that might get fixed. Mail server out of disk space, you're greylisted etc. So try again later. Those can take a few days to give up retrying.

Ah, but you forgot to send it from a Gmail address! That's the big important requirement!

> Refusing it outright is a case for the European Commission which hopefully will slap you another few billions of fine onto the wrist until you remember to play nicely with the other kids.

This is a very interesting idea that I completely missed.

> There are going to be false positives, we will make mistakes, but we certainly care a lot about fixing issues like this when we hear about them.

Oh really? Then what is the mechanism by which an affected user can report an issue like this and receive individual, accountable response? Because that is what a company that cares about fixing issues does.

As far as I can tell, the only redress available for any issue like this with Google is 1) be a big enough name like Jamie Zawinksi (he has been posting about struggles with this exact issue for several months) that your blog post gets attention, or 2) hope your blog post makes the front page on a site like HN.

Why didn't you link JWZ's blog post?

b^)

His blog is very easy to find, but also is full of all sorts of off-topic NSFW stuff which I don't really want to send people into unawares if they're not familiar with who he is. He has several recent posts on this topic which I don't feel like tracking down as their content doesn't really add anything to the conversation -- it's basically exactly what you see in the OP here, with more colorful expletives.

This is the part that's going to bite them when antitrust comes around.

They are largely unaccountable and there's no real way to get a hold of a human being unless you shame them on social media and it gets enough traction.

If you're a paying customer, you definitely can get a human being to look at your problem.

The difficulty in this case is the people operating SMTP servers are not paying customers, the free Gmail users are not paying customers and the GSuite paying customers don't think it's their problem (after all, the spam filters works quite well these days and lots of people are on Gmail, making the problem very small in the receiving end of this situation).

It seems an external force will have to push Google in the right direction (of having systems in place to deal with the exceptions). If that's going to be public outcry, legislation, etc... We will see.

On the automation front, it seems Gmail could do a better job at tracking sender reputation over time (i.e. you haven't send spam in a while, we'll be more lenient with our spam rules).

> If you're a paying customer, you definitely can get a human being to look at your problem.

Doubt. We're paying ~$10k/month on ads. The Ads API is rate limited, but it's either broken or has arcane logic, so about every 4-6 weeks, the usual workload will trigger it and we're shut out for 12-24 hours. Can talk to ad support whenever I want, but all they are good for is optimizing campaigns and explaining the UI. No option to get to technical support, no option for ads support to escalate.

Of course, we might not be a large enough customer, but it certainly isn't about "paying".

Here's the form to make a complaint about customer support:

https://support.google.com/google-ads/contact/aw_complaint

Thank you, but I and pretty much everybody at the company has given up months ago. We've tried everything we could (well, I guess we should've tried to go viral on HN or twitter...) and it got ignored again and again.

It has become a bit of a rite of passage when somebody new arrives, they see the error ticket for the first time and are eager to dig into it and solve a long standing problem. It's like a trust fall, only the lesson is not to trust, and you learn that lesson by falling.

>If you're a paying customer, you definitely can get a human being to look at your problem.

Based on the number of hacker news posts from people who pay them money and are locked out of their accounts by automated systems with no recourse I'd say that's utterly false.

Maybe you meant to say "if you pay them enough money" which is true. But most people don't have that much money to pay them.

I'm not saying Google's support is stellar but there's a lot of misinformation going around. It's become a meme to say you can't reach a human at Google, they will shutdown new apps after an year, etc.

Have you had direct experience with Google support? My experience has been pretty regular but maybe I'm an outlier.

Recently we recovered a GSuite account that was still in the trial period! We hadn't even entered billing info :)

The process took a while (about a week) but we did get the account back.

Things have changed a little bit since Google just presented a wall for support requests to bounce off.

> If you're a paying customer, you definitely can get a human being to look at your problem.

I may not be giving them money directly, but my use of their products contributes to their billions in ad revenue. Without us free users providing eyeballs for their ad network they would be significantly less rich.

It's upsetting that the default response to complaints about Google's terrible customer service is "if you don't pay them you aren't a customer." Most of their value comes from us, it most certainly isn't a one-way relationship.

… or 3) "if you can't beat them, join them!"

It feels like Google hopes that you just give up on self-hosting your own email and simply move over to Gsuite. It certainly feels that way. Especially when it feels like Google/Gmail/Gsuite are commonly recommended nowadays.

Glad you're here and listening. I'm another poor schmuck who runs his own email server for family members (you can look it up from my HN profile -> mx record) and Gmail deliverability has been a problem for me for over a decade. I do DKIM, SPF, reverse DNS, had the same IP for many years, etc etc., and it's a total crapshoot whether Gmailers get my message in their inbox or spam folder. Postmaster tools have been useless for me because apparently my volumes aren't high enough.

I'm inclined to think Gmail's approach to this problem is fundamentally flawed, because Gmail has been by far the worst at binning my server's emails. I don't have this problem with Hotmail, AOL, Yahoo, etc., etc. Now, feel free to argue that their approaches are too lenient, but if I have to choose between false negatives and false positives in my spam filter, I absolutely know which one I'd choose.

> Gmail does not, inherently, have any network effects

This is wrong. The current situation is "if you want to reliably contact gmail users you need a gmail address". This is no different than Facebook or messengers. And that may actually be the incentive for Gmail being such a bad player compared to everybody else.

(also @reaperducer) Something I didn't fully appreciate until I got here was: 1. Work (aka enterprise) sends much more email than consumer, and 2. Outlook dominates enterprise. Their market share is pretty astounding.

I assure you there is a lot of email coming from Outlook/Exchange.

My employer is one of those enterprises; we host our own Exchange server on our own hardware on our own IP address.

I suspect that is not super relevant to this conversation, because there are a lot of signals that Google can use to see that we’re a big fish, and therefore automatically tread more lightly on the SMTP bouncing. We’re running Exchange, for one thing, which is not cheap or easy to use. We’re in the IP address of a big enterprise ISP. We have other enterprise services running on adjacent IPs, like OWA and websites.

Small personal email servers have none of those sort of “ambient” signals; they’re probably running open source email server software on a single IP coming from a consumer ISP or general data center IP space.

So the question might be why Google seems to react more to those ambient signals than other email providers. Because we have no issue at all getting personal emails into a Gmail inbox.

It doesn't feel like "... you need a GMail or an Outlook address" is much of an improvement.

Also, AMP for Gmail only works in Gmail.

https://www.blog.google/products/g-suite/bringing-power-amp-...

This is unfortunately technically untrue, as Google has beaten Outlook and Yahoo and Mail.ru into submission and into supporting it. It's still a huge security risk, and an inherently proprietary email spec Google designed and implemented without community input and deployed despite significant community resistance.

But technically, it works outside of Gmail.

> as Google has beaten Outlook and Yahoo and Mail.ru

Microsoft and Verizon are hardly vulnerable timid players.

Perhaps the solution is to have two e-mail accounts: a GMail account to communicate with GMail users, and a non-GMail account to communicate with non-GMail users.

Ideally, also a client that handles all this automatically, i.e. chooses the right sender account depending on the recipient, so that I don't have to think about it.

Just like Pidgin was a common solution for AIM and ICQ, this would be a common solution for e-mail and GMail.

As a "small fry" myself, the hoops I've had to jump through in order for Gmail to accept legitimate mail from my mail server are rather large. Like the op, legit email appears in the trash of some recipients for goodness knows why. Like OP I've spf, dkim, & have never been on rbl. I (& services on my machine) are the only senders, and the recipients of those machine messages want them (about 10ppl in total & email have no marketing content). I communicate on port 25 using tls where possible.

It's only ever Gmail that sends mail to spam. Every test I run on that email marks it as clean (generally spam assassin)

Gmail is certainly NOT a part of the open email system you mention, but is a constant thorn in open communication.

Ironically, I use Gapps for one domain and because of my spf settings with "include:_ghs.google.com" I get every cat and his dog trying to send as users from my domain, which thankfully end up in that domain's gapps spam. (hint: let us use geo located includes, like _ghs-us.google.com or _ghs-au.google.com, so that there's a smaller include list!)

I have a small mail server and fortunately, as long as my server is well configured gmail has always been well behaved for me. Microsoft, on the other hand...

Secondly, we care deeply about having positive relationships with developers and all our users.

This is so deeply disingenuous and at odds with so many peoples experience it’s absurd. Or satire?

Probably key point is "..with developers and all our users.." but not individual developers and individual users.

Key point is that words don't matter unless they're backed up by some concrete action.

The worst thing about this blocking is that you often times don't get to see what's wrong. So you're running around in circles, trying to use various mx tools on the internet to find the problem.

This is not just gmail issue. Though with many smaller providers you'll at least get something like "your blocked by XYZ blacklist".

> This is not just gmail issue. Though with many smaller providers you'll at least get something like "your blocked by XYZ blacklist".

I would much rather get this than silent discard. At least when I've seen things in the logs in the past (yes, I do read my server logs), I have been able to follow up and fix them. I get the sense that Google is even going beyond using 550 and discarding email silently. I just did a grep through my logs to see if this message came up and found it nowhere, yet I have users on my mailing lists claiming they don't get emails and they've checked their spam folders. They're on GMail.

I think (some of them) may think that's true, but their internal reward structures and top-down priorities are clearly set up such that it's not, in practice. There's no other reason it could be so bad.

This seems to be common with the product teams I’ve interfaced with. Good intentions, but bad prioritization and little incentive to do anything other than cater to the big revenue drivers.

Why would you start a for-profit company to not focus on "big revenue drivers"?

If you are a for-profit company, why would you kill off business lines that are "only" making you $10-$50 million per year?

Most likely due to opportunity cost. Your resources and effort would be better spent elsewhere.

Immediate profit and cost saving might hide something more valuable to a business in the long run: reputation.

I live in Argentina. Few years back, an ex government official from the previous government was caught in the act of hiding 8 million dollars in a church.

My partner's auncle was also an ex government official from the previous government.

He was in charge of the country's office for regulation work in rural areas.

Up to that point, he was a complete believer that the government he was part of, was there to really change things and put the country in the right track.

When the news about that other guy's 8 mill broke out, he was completelly and uterly devastated.

I think the same can be applied to a lot of the people that works in Google regarding the way they perceive the company.

I just recently read in the news that there has been some retaliation exherted on googlers because of organizing some walkouts.

To think that your company just does good because you try to do good, and not take a look at what is really going around in the world related to what the company you work for is "a little" naive.

Also, I think that right now companies like Google and FB should be treated and reasoned about like states. To think that they will not abuse their power in one way or the other - just like countries do - is also "a little" naive.

>Also, I think that right now companies like Google and FB should be treated and reasoned about like states. To think that they will not abuse their power in one way or the other - just like countries do - is also "a little" naive.

That companies this big ought to be reasoned about just like Nation States is a very interesting idea.

Alternatively, we could reject the notion entirely and revive the antitrust movement. Centralized private power is just as, if not more, dangerous than centralized public power and we aught to have a discussion about whether nation-state size companies are really a desirable feature.

I think both the approaches in your comment and the one above (Or any comment to that matter) are worthy of a lengthy discussion.

But my point was more directed towrds understanding how companies this big are starting to behave.

I do think antitrust is a way to look at it, but the anti trust laws in the US I think are difficult to point towards Google for instance (Probably the same about European laws)

> we aught to have a discussion about whether nation-state size companies are really a desirable feature

I think it goes a way longer than that. Obviously private companies are subjected to a lot less scrutiny than states, but states do miss behave and do not run into the risk of eternal intervention, unless they really fuck up, and even then.

Forcing mega corps to assume the responsibility of statehood is one of the key plot points in the "Poor mans fight" series of books. (some of the best sci-fi I've read in a while too).

Hey, thanks for that. Interesting. I'll take a look at those books.

As I started to read this, I SOOO wanted it to continue on like this:

> I have this 8 million from the church, but I need to get it out of the country in order to spend it ...

Why stop there, and not go full nigerian scam, Argentinian style.

How many people don't experience problems sending to gmail? Do you think blog posts criticizing and praising gmail deliverability are in equal proportion to users' experiences?

I don't really care about how many people it is, I primarily care they are fucking up my mail, with absolutely zero recourse or transparency whatsoever, just a total black hole.

I'd be surprised if there are any self-hosters who never had any of their emails ending up in Gmail's spam folder.

Disclosure: I've been self-hosting for 5 years.

Until recently, I didn't think I have a problem with GMail spam filtering.

Then I looked at what was in my Spam folder. I haven't looked into it for a long time before, because, well, Google has trained me that their spam filtering "just works".

I now try to check it at least once a week. The amount of false positives was staggering, and some of it was important.

Agreed. How exactly do external developers reach out to google?

I had a similar reaction to that statement. Google tends to make the happy path quite pleasant, but the moment you stray from it, things get kafkaesque very, very quickly.

Saying this from firsthand experience as a customer of Google's B2B offerings, Google earned its bad reputation on the enterprise side.

Saying this after a good number of years as a PM: when you spend 40-50 hours a week thinking about your product, it's really easy to get lost in the weeds and forget that your users see your product from a totally different perspective than you do. I see this all the time because it's very, very difficult to avoid this problem.

If we lose an open, healthy ecosystem with many providers, we'll destroy the base we stand on.

Or you've grown large enough that the base no longer matters.

we care deeply about having positive relationships with developers and all our users

it definitely makes me sad to see articles like this

There are going to be false positives, we will make mistakes

we could do much better there.

Most of this reads like it was written by Facebook's PR department after yet another scandal.

Ooopsie, that's another oopsie. We'll fix it, now fuck off underling..

> Gmail's success is entirely predicated on the health of the global email ecosystem

That depends on what you define as "health".

> Email itself, of course has a huge network effect, and that is because you can email anyone in the world, regardless of what email system they use

If that's the case, then why does Gmail system explicitly thwart this objective, as described by this very article?

> If we lose an open, healthy ecosystem with many providers, we'll destroy the base we stand on

And that's exactly what Gmail is doing when it does what is described in the article.

> we care deeply about having positive relationships with developers and all our users

You personally might, but your company does not. What your company cares about is advertising revenue. If your company really cared about users, it would figure out a way to let them pay directly for your company's services (not just Gmail but search, maps, etc.) so you could see directly from users how valuable those services were, instead of having any benefit to users be a side effect of trying to capture their eyeballs for advertisers.

I agree completely with everything you said.

It's too bad that Google has no incentive to allow a premium/ad-free experience as that implies having ads is a negative thing, which basically undermines their entire business model.

I would pay quite a lot for google-class services with a binding contract that limits their options for spying on me and is 100% ad free. Instead, I spend an inordinate amount of time configuring my computers to avoid accidentally using their free “services”...

They have G Suite, and they also tried a more consumer oriented version but no one wanted to pay.

I would love it if more of the Internet were based on payments and micro-payments but consumers have voted decisively against that.

I try to sell people on the idea of paying for news but extremely few are willing even for something that critical.

> consumers have voted decisively against that

No, they haven't; they've been given no opportunity to vote. Such an opportunity would be, for example, Google offering its basic search service for pay to users. Of course they would have to offer some incentive, some benefit that free users (more precisely, ad-supported users) don't get, but we already know how that works: the obvious benefit is no ads. I would certainly pay for an ad-free Google.

> They have G Suite, and they also tried a more consumer oriented version but no one wanted to pay.

I have paid them for years for photo storage. Looking for somewhere to move it in case their AI suddenly decides it doesn't like me (I've already went through a reCaptcha party after trying to dig up some documentation for some special error messages that I felt should be out there somewhere.)

> I would love it if more of the Internet were based on payments and micro-payments but consumers have voted decisively against that.

Didn't have a chance at all to vote.

They could have earned a lot more on than than than they do from me not clicking on the dumbest ads I know about.

> They could have earned a lot more on than than than they do from me not clicking on the dumbest ads I know about.

Then they would have - they are a for-profit company.

My assertion that consumers are not willing to pay if they can get something 'free' (ie. with targetted ads) is hardly new or controversial.

How many news sources do you pay for? I pay for my news and take a very keen interest in its quality, but the media outlets have an extremely difficult time convincing enough people to care enough to pay even for their news - the most important thing of all.

>> They could have earned a lot more on than than than they do from me not clicking on the dumbest ads I know about.

> Then they would have - they are a for-profit company.

I doubt they ever considered me as a person, only as male 25-65 => show ads for scammy dating sites.

> How many news sources do you pay for?

2 newspapers, +used blendle a while ago, ready to pay for more when I can pay pr read.

I also have paid other things like an tech/art channel on a video site etc.

(I guess I'm not the only HNer that does this?)

> but the media outlets have an extremely difficult time convincing enough people to care enough to pay even for their news - the most important thing of all.

Around here that might be because my choice is either a two hour drive to somewhere that sells that paper printed, -or to sign up for an auto-renewing subscription.

I could of course sign up and cancel but I already has too much on my plate (more than two kids, trying to be active in my communities etc.)

> Firstly, Gmail's success is entirely predicated on the health of the global email ecosystem. Gmail does not, inherently, have any network effects (unlike FB, messengers, any other comms tool of Gmail's scale)

Google initially federated with XMPP, until you had critical mass and shut out the rest of the Jabber/XMPP ecosystem.

Of course now you seem intent on killing your _own_ messaging product so it's hard to divine any self-consistent intent here.

> we certainly care a lot about fixing issues like this when we hear about them.

And there's the problem. The only effective way for OP to contact someone who likely has the ability to fix it was to write an article, post it on HN, get upvotes, and hope you happen upon it.

That's not a viable solution for everyone who runs a mail server Gmail falsely identifies as a source of spam. You don't have a good way to hear about issues like this.

> I'm one of the PMs for Gmail and hang around HN quite a bit.

Can you bring back the old web UI?

I experience obnoxious interaction and interface bugs in the new one on an hourly basis, it has horrendous interaction latency for many operations that used to be snappy, it looks worse, and has basically nothing new I want.

Actually, can you bring back the Gmail of like 10 years ago? Basically every change since then has been negative for me.

If you install noscript and log in the old UI comes back and it's amazing! I've actually stopped using mutt on my laptop for gmail because of it.

Stopped using mutt! Strange!!

New UI really makes Gmail barely usable, basically on any kind of connection. Since it came into being, I avoid checking gmail because its unbearably slow and buggy.

I have 20 years of XP in this industry and build large gov services, and I am astonished that Google allows itself such a failure.

If you want, I will make a gif of Gmail unresponsiveness in default Chrome browser on 50MB connection - taking more then a minute to load and then some more to stop glitching. Then I will show you ProtonMail, Zimbra etc. having no problem whatsoever.

Its embarrassing and affects all their cloud tools in some measure. Google was once a good company. Today, most of the IT people I know or work with try to replace anything google.

Above post from the google employee which "cares" makes me angry - what do you think m8 ? That we are squirrels ? Shame on you. If you have some moral decency that you propagate here, go and work for somebody non-evil, take as many people you can with you and write a blog about it so anybody can know.

Just use the "basic HTML" view. When the new design is loading, there will be a link at the bottom to click to use basic HTML, and then there will be a button to set it as default. I use this and have (almost) zero complaints. It is an excellent mail client.

Was never fun of 1995 Internet design really. I do use it, but it looks terrible, organizational capabilites are limited and ... did I say it looks terrible ?

Not to mention it always returns to default view, so you need to click html view each time.

I know its not politically correct and I am sorry that some people will have their feelings hurt, but lets pretend that we speak to grown ups for a moment and not 13 year old teenage girls: the GMail is gigantic pile of shit and due to the fact that personal or other non-shitty mails will be blocked because they do not fit Google agenda, there is really no easy way out of this lock-in. Making it euphemistic and diplomatic counts in my world as dishonesty so if you do so, go freck yourself too because you are contributing to destruction of this nice planet we live in.

At least I have freedom to express and say here that Google can go fuck themselves (minuses are welcome, counting morons is my interpretation), and please Google PMs, take those words as my default response any time when Google Something asks me on a phone 'Hey, you were at XYZ, how was it?' during those 3 seconds I keep GPS on. Please make default answer - go fuck yourself, thats how.

>Firstly, Gmail's success is entirely predicated on the health of the global email ecosystem. Gmail does not, inherently, have any network effects [...]

That's ... unconvincing to the point where it sounds like PR spin. How does Gmail not benefit from crappy email service outside of gmail? How do more gmail users not accentuate this?

>Secondly, we care deeply about having positive relationships with developers and all our users.

Frankly, talk is cheap.

I don't mean to be rude, but your response is either (a) completely missing the point or (b) disingenuous. Assuming the former, can you substantiate your claims?

Hey, thanks for replying. Just sticking your head over the parapet is pretty brave.

I have a contra story. One of no delivery issues to Gmail or Hotmail, none, zero, nada. I've run a private email server (friends/family/small business/private mailing lists) for two decades. It's kept pace with every possible factor for reliable delivery - SPF/DKIM/DMARC/ARC, valid client SSL, IPv6, correct PTRs, DNSSEC etc and have no sketchy affiliates. In that time the IPv4 address changed exactly once and has never been RBL'd. Our mail gets delivered AOK.

And yet, even though I think my compliance level is good, I still feel like the blind man groping an elephant. I'm hoping I'm perceiving things correctly; I have no idea if I'm missing something. It helps that I'm an old-school ISP engineering inmate and contributor to well-known MTAs and MDAs, but few folks are lucky enough to have exposure to so SMTP radiation.

My take on the Postmaster Tools is that they've been created entirely to serve Google's purposes, and thereby serve no-one well because (as you point out) it's ecosystem engagement that makes a difference. If you sincerely have an incentive to improve, there is an awful lot of work to do there. It's okay to push the burden of compliance back to the sender, but the Postmaster tools offer only the most rudimentary levers to pull and provide almost no useful information, particularly for smaller/indy senders.

The message that comes through is that Google only really gives a shit about other large scale entities and struggles to see other points of view. This stands in quite stark contrast to Google's effort level over HTTP certificates and webmaster tools.

Thanks for wading in.

"Gmail does not, inherently, have any network effects" - you do and you can't help it.

I've been doing email for quite a while (20 odd years) and I don't find Gmail refusing my customer's email for silly reasons too often.

I have had some odd rejections from Gmail (int al) and no-one to talk to. You opine that Postmaster tools are under-invested in but actually miss the real point:

You (G) seem to under-invest in people and put too much faith in magic (AI/ML/nonsense). I really am not a Luddite (I'm giving the tyres on my smart new Node-RED home IoT thingie a good talking to via Javascript right now) but please don't forget ... "memento homo")

Small anecdote: The acceptance email to my universities study abroad program landed in my Gmail spam folder. An email that I had 5 days to react to or someone else gets the spot.

I thought it's great that I have so rarely stuff in my Gmail spam folder that I don't even have to check it regularly until I nearly missed a very important email. Apparently I was lucky that ot didn't get outright rejected.

I rather have to deal with some spam than to miss important, or even regular, emails.

I use email hosting from my domain name provider (email in my profile). I regularly find out that personal emails from me end up in people's spam folders on gmail. As far as I can guess from digging into email headers, gmail is upset because my domain name provider's email server doesn't use whatever the latest mail-server signing mechanism gmail expects is. There's absolutely nothing I can do about that, and no obvious way I can get gmail to stop treating my mail as questionable and throwing it in people's spam folders. And it's not at all obvious how to debug this.

I get why gmail isn't transparent about all their spam-filtering mechanisms. But when personal email to gmail users is being regularly classified as spam, that produces a serious usability problem for non-gmail users.

>Firstly, Gmail's success is entirely predicated on the health of the global email ecosystem.

It absolutely does not. It is so large now that what the parent poster said is the real case. It has such dominance that people are pressured to switch to Gmail or Google apps to get stuff reliably delivered to the massive percentage of email users.

This is exactly like an IE6 PM coming on extolling the virtues of open standards and how they are critical to the success of internet explorer.

Seriously think about it (assuming you are even posting in good faith). If Gmail's success was "entirely predicated on the health", do you really think Postmaster tools and other interoperability efforts would have so little investment?

> Firstly, Gmail's success is entirely predicated on the health of the global email ecosystem.

That simply is not relevant anymore if 99% of email is on Gmail.

I want to echo the exact same issues as the original author. Running my own email servers since the 90s I recently just “gave up” - no single indication about spam issues. All green on several tests. But Google rejects mail. Zero help or tools from google to get off their filter or whatever it is that makes these kind of decisions.

Friends and family all have Gmail accounts for those important emails that Must go through. Enough said.

I’ve moved my email to another provider and stopped self hosting.

> That simply is not relevant anymore if 99% of email is on Gmail.

He says elsewhere that more email is sent via Exchange servers than GMail.

>> but we certainly care a lot about fixing issues like this when we hear about them

It's sheer chance that you happened upon this HN post. Google needs to have an open, healthy customer service infrastructure in place to hear about, track, resolve, and follow up on reported issues. This is antithetical to "The Google Way". That is how you get posts like this.

Gmail stopped being a good email provider years ago:

https://github.com/dominictarr/your-web-app-is-bloated

Gmail used to be a small, light, fast and simple email service. Frankly, you should be ashamed of what Gmail has become.

But then, how can a simple heuristic like "hey, this guy has sent and received messages from this non-google address before, we should most definitely not drop incoming mail from there" not be in place? I gmail is pretty much the only google service I still use. In pretty much all regards, google has repeatedly disappointed me over the last couple years. A couple months ago, gmail suddenly started dropping incoming mails from a mailing list I've been subscribed to for 8(!!!) years, three times so far. Exactly like in the linked blog post here.

To sum it up: sorry, but what you're saying here doesn't sound like it's true, and if it is let me tell you that you're failing at it, miserably. The only reason I didn't move away from gmail is that it's just a lot of work. But the time will come, eventually.

My properly configured DKIM/SPF/etc mailserver, with a 10-year old IP, still gets email redirected to gmails spam folder more often than not. Even replies to emails from gmail addresses, with reply-to-id's.

That's the box I use for my personal email. Out of principle. I believe in a distributed/federated net. It just means that I can't expect that emails I send actually get delivered. Sigh.

My experience for business has been that you simply have to bite the bullet and be on outlook.com or gmail.com. Or you can't expect that regular email always arrives. What everyone in the corporate world expects of course.

I've heard this story from everyone I know that operates their own mailserver. No hyperbole. I would be tremendously happy if Google would spend some serious (and visible) effort on this. Thanks Paul.

Keep trying to make yourself feel better. The company you work for is ruining the internet which means you are complicit in just that. Perhaps you could find something more constructive to do with your life.

> Gmail does not, inherently, have any network effects (unlike FB, messengers, any other comms tool of Gmail's scale)

That's not true at all. I've noticed when sending from Gmail to Gmail it always gets delivered and quickly. I've noticed with other email service people, even ones using big providers like Yahoo, it's really hit or miss if they can receive from me, send to me, and it takes a long time to show up in my email. So there's a really strong network effect to use Gmail because Gmail doesn't work well outside Gmail. So much so that I keep a bunch of address (personal/Gmail, work, school, my personal web site and email server) and try to pick the one that works best based on who I'm sending to.

Every single person inside of an institution can care deeply about something, but that doesn't mean the institution as a whole will act as if it cares. Think of Google as a huge animal made up of individual people. The people can perceive and act on their immediate surroundings, but only by forming larger structures (like teams or orgs) are they able to act on the world at Google's scale.

Really, it's not about incentives; it's about perception. Just like you can't act at Google's scale on your own, you can't perceive the effects of Google's actions on your own. You care about fixing issues when you hear about them, but how do you hear about them? What can the Google-animal see, and what is it blind to?

Actions speak louder than words. When I see generic PR phrases like "we care deeply", my bullshit detector immediately goes off the wire.

I very much want to believe that Google cares about small email providers and self-hosters. I believe that you care personally, but have a look at the documentation. All the help articles target bulk-hosters, the documentation doesn't even mention humans! The case that a human wants to send an email to another human, which is the very reason email was created in the first place, is not even covered or planned, so I'm pretty sure Google doesn't care about collateral like us, who have fun caring about their own email servers, playing by the rules and doing everything to not be on blacklists.

Why not count the amount of emails coming from those domains and give them a "softer" filtering algo if it's below a certain threshold? Or open a whitelisting program where you can go full Google on people who violate the terms? I think there are (automatable) ways to solve this problems, but only if Google understands that emails are for human interaction, not to receive ads.

(Edited for typos)

I'm someone in exactly the same situation as the author of this blog post. I appreciate your well-meaning response, but the bottom line is that Google will have to offer a way for a real two-way communication channel for email administrators or be considered as acting in bad faith.

There needs to be a way to inspect what caused mail delivery so we are able to fix it. There needs to be a way to provide feedback to Google about mail that was wrongly rejected. If you can push this forward internally, please do it.

This is a far too regular occurrence to just ignore: it is a rule that delivering email to gmail from outside is unreliable, not an exception.

This is your personal opinion. And there is the experience of those people out there. I believe you that you personally stand for these but I seriously doubt your employer cares. There are many examples of the same issue, like that one with Firefox.

https://www.zdnet.com/article/former-mozilla-exec-google-has...

It is kind of funny to me that other email vendors could implement SMTP and co properly and Gmail _more recently_ started to have issues. Just because of this I started to look for alternatives because I will not stand and watch how an ad-tech company destroys the open internet, even if this will be very inconvenient personally.

Wow since you're so concerned and helpful, here's a tidbit: Everyone who has tried to run a personal mail server in past five years can't because of gmail.

N=1, but my small email server actually hosts emails for a business, and the only problems I've ever had were with Microsoft, and they went away after i jumped through some hoops for them. I'm using a linode and dkim/spf/dmarc/tls/rdns.

how do you tell the difference between a personal mail server and a spammer?

It's not really that hard.

Personal email server will send ~100 emails a month (maybe 1000 if they are extra prolific). Roughly half of those emails will be in reply to emails that originated on your mail server (you can link those with basically unique message-ids that you generated). There will be a bunch of extra safety measures on the server (reverse DNS, SPF records, no open relay...).

Spammers, on the other hand, will send thousands of messages every hour. And have none of that "other" stuff.

Spammers can buy cheap VPS and setup all this stuff with scripts. So I don't think that it really helps. It's good to have it (to prevent spoofing), but I'm not even sure that Gmail will use wrong SPF record to deny e-mail, it's not really part of SMTP, those are just optional additions.

But volume is a good argument. I don't think that spamming could be cost-effective with something like 100 e-mails per day from a single IP-address. So why not just let small servers pass every filter in the world (may be except reverse-dns record) as long as they are small? The danger should not be very big.

Of course I might be missing something, I have no idea how spammers really operate. Or may be Google just don't care about personal servers at all.

You can't automate your way out of paying for a domain, too, and that costs significantly more than an IP.

one would think that Gmail would be smart enough to not flag a reply to an email sent from a Gmail user as spam, but no. it flagged it. :/

So apart from the hang-wringing going on here. What can the original poster do?

They have literally shown what they have tried to do and it is not enough.

If this poor guy is a "false positive" then what is his path to resolution? It looks like he doesn't have one and saying "Yeah, that sucks" or "Because you made it on hacker news we will investigate" isn't helping all the other "small fry" out there.

Thanks for the thoughtful response.

But the problem with all Google services is that there's apparently nobody listening. Unless you're a paying customer, and even then, I gather that it's iffy. It's all automated, in extremely unhelpful ways.

I do appreciate, however, that it's largely a scale issue. Google is just so big, and operates on such small margins, that it's arguably not economically feasible to provide individualized support.

Unless you make it to the HN front page, or whatever.

As a practical matter, if you want to send email to Gmail accounts, you'd better be using Gmail. And apparently, even that doesn't always work.

Thanks for sticking your neck out. When these kinds of articles happen, it often feels like a witch hunt.

How extensively does the Gmail team test interoperability with other mail systems? From my personal experience, one sin committed by many teams who in say they play nice with the ecosystem is to neglect to test integration against other players in the ecosystem. It's annoying to test and slows down development pace, so frequently no developer wants to do it. This leads to a situation where it may be unintentional, but there is definitely a "favored client."

It really got worse. In the past years (always the same IP, good reputation, DKIM), I had a few mails land in the spam folder of gmail users, but it did not happen very often.

This week I encountered the following situation the first time: 1. I received an email from a Google apps (or whatever it is called) user. My response to that email landed in the spam folder. 2. I exchanged one or two mails more with another person in the same organisation without problems. 3. Suddenly my response got rejected. I tried sending via various means such as via Thunderbird, via mobile on 4G, via mobile on Wi-Fi, changing the content slightly... No chance. My email got rejected. In the I ended up calling the person from number 1 on his mobile to get the second person's mobile number... Needless to say, I have been less than pleased.

Person 1 was very surprised about their spam filter misbehaving, but at least I can now offer an excuse by pointing to this blog post...

> but we certainly care a lot about fixing issues like this when we hear about them.

What then should we make of the fact that Google practically ensures that they won't hear about such issues by making it next to impossible to report them?

How can you seriously argue you have no network effects when Gmail has repeatedly added (to be fair, somewhat useful) features that basically only work on Gmail, like AMP emails or expiring/deletable emails? Sure, other vendors could go out of their way to reimplement AMP and shove it into email but really?

I'd love to have a discussion about gmail but offline, feel free to mail me. I've just transitioned a small company over to gmail and the experience so far has been nothing short of terrible, this can't be how it is meant to be. If you're up for it my email is in my profile.

I'm glad you have your heart in the right place.

However, as one of the PMs in charge of one of the most important messaging applications on the planet, you cannot claim the following without stating your reasons:

> Gmail does not, inherently, have any network effects

Yes, compared to facebook, and within the context of messaging apps, gmail's network effect is weaker. But the fact that the very existence of the gmail monolith can kill off small mail servers (not implicitly, but by failure to deliver messages), is a real problem.

I understand it's a difficult problem. It's not a gmail scale problem, but the effects, times the number of small mail servers is important, because those small mail servers will be what is left if any of the big players in this ecosystem collapse.

"It's because email is open. If we lose an open, healthy ecosystem with many providers"

Don't you see? This post on HN means you have created the CLOSED unhealthy ecosystem. You built what you just spoke against when you shut out email from small providers. Shame on you for speaking idealically and not confronting the reality of the situation.

Are there metrics regarding this ecosystem that Gmail is optimizing for and which Google engineers can get rewarded for improving?

I've had emails sent to gmail users from our paid MS Exchange accounts blackholed. And not just randomly once or twice. My wife was a contractor for someone and despite having years of previous conversations with them, one day they just stopped getting her emails altogether. She had to create a gmail account just to keep working with them.

You are a PM, you may “theoretically” care when you see something in HN or some other social media, but the number of people having problems with your product are magnitudes larger than that. Everything is completely filtered out by customer support, even for companies that have good customer support unlike Google, and never reaches your ears.

I am not taking a stab at you personally. I truly believe that you care when you read the story here. But if you really cared enough there would be no story to be reported here.

Have you guys considered an automatic system with collateral. I'll gladly sign a contract stating that I will be under X% bounce rate for the next month and put some collateral (depending on the email volume requested). At the end of the month you get your collateral back.

It's a purely economic and automated solution that should work well enough for small operators (collateral for a small email volume should be small since spammers really need high volume)

I call bullshit. Google has flaunted their complete disregard for RFCs time and time again. Try to send an abuse complaint to abuse@ any domain that's hosted by Google. You can't. Try to correspond with a human. You can't.

Content filters should never be applied to abuse addresses, yet Google happily ignores this and never even replies to abuse complaints sent to @gmail.com or @google.com.

> Gmail does not, inherently, have any network effects

If you reject most non-Gmail messages it does!

Inventing a reason to reject mail does not sound very much like “caring deeply”. More like “let them deal with it”.

If this is your personal take, then who is that "we" you keep saying?

> Gmail does not, inherently, have any network effects

In an ideal world, yes. Unless Gmail makes it harder for anyone using a different provider to communicate with Gmail users. In which case you've effectively "encouraged"/forced everyone else to use Gmail to take advantage of the Gmail network. This seems to be exactly what's happened to the author. This is very similar to the complaint voiced recently by the Mozilla leader as well.

I don't doubt that you as an individual have very good intentions. But individual intentions do not translate well to organizational priorities. The best way to prevent Gmail-lock-in in the long term is to have more diversity and competition in this marketplace.

    we will make mistakes

Completely understandable.

What I don't understand: why aren't these false positives sent to the recipient's Spam folder? Why else does that folder exist?

NB: Google absolutely has network effects, and those are to an nth approximation its entire value.

it's cool that you hang around HN, but this kind of response just exacerbates the big problem with Google these days: unless you can get to the top of HN, or picked up by TechCrunch, you have no hope of getting any resolution for any of the bad things that Google does. Whether you have incentive to fix it or not, the reality is that you refuse to acknowledge these "false positives" or give people any recourse to fix them.

I appreciate the response and the evident care at the personal level that you apply to this.

It is also an excellent goal to use AI to scale as many processes as practical to provide ever-broader services.

Yet, it is obvious that, at the corporate level and despite being one of the most massively well-funded companies worldwide, Google is famous for abysmal customer service and most particularly, providing NO way whatsoever for a person who has been damaged by a "false positive" to every correct a real person at Google to get a resolution.

Yes, providing an avenue to real people who could resolve the issue would be expensive. I seriously doubt that it would be so expensive as to make any serious dent in Aplhabet's profitability or stock price (unless, of course, the problems are far more massive than anyone knows).

Moreover, providing the ability for people who have been "false positive" damaged by some spam, account violation, etc. to reach a person for resolution to a problem would also provide large amounts of the fine-grained detailed data that would allow Google to fix its issues at the algorithm level, thereby reducing the need for the support service as well as the frustration.

Why is this not done? Is it that the problems are actually massive but hidden? Is it that management does not care the way you do? Is it that mgt actually wants to drive off all small services in order to dominate? Obviously all speculation, including bad speculation. But leaving zero ways to contact, as well as zero information about the issues, leave the field ripe for all the speculation. This is the sort of reputational problem that can fester for years, seen only as a minor issue -- until it grows so massive and passes a tipping point, where the reputation and market position is unrecoverable. I hope Google does not go that route.

If Google really cared about the health of the global email ecosystem maybe you guys should consider putting a GPL on Gmail. Maybe if we could study the code and come up with a solution that benefits everyone maybe we could make email better. Instead you guys at Google just want to maintain your near monopoly on a vital part of our online existence.

Just signed up for two newsletters today, both “Please confirm your subscription” messages went to spam. Corrected it then hit the confirm links. One “subscription confirmed” email went to inbox after that, the other still went to spam.

Guess I’ll find out whether correcting it a second time applies to the actual newsletter messages.

Thank you for the reply. Wouldn't it, somehow, be possible for Google to provide a place where one can go when such problems occur? Maybe paid (in order to scale). As is one feels completely powerless and negative emotions will be caused.

>Secondly, we care deeply about having positive relationships with developers and all our users

I bet you also take the security of your system seriously. What other Internet platitudes can you spout?

Google the giant will only bend when lawsuit happens. Assuming Google's AI put an urgent email to spam by mistake and made a huge loss to some party, Google should be immediately sued, probably by class action, only then it will respect small and well-behaved players like the original post here. Whenever I read post like this, I recall its once popular(now withdrawn) slogan "don't be evil".

Email is not an appropriate method of communication for highly urgent, impactful, and time-sensitive messages. So the party should sue themselves for negligence.

There are frequently situations where the email is very impactful to the recipient, much less so to the sender.

Secondly, we care deeply about having positive relationships with developers and all our users

Holy shit, GMail's going away? This is the kind of thing we're told just before a Google product is killed.

GMail is placing email from other users in the same google suites domain into spam which makes 0 sense since google knows they all belong to the same gsuite account.

Also, a lot of big companies don't even allow their employees to use Gmail, but I suspect they are allowed to send email to people on Gmail.

So will you fix OP's problem then, prlambert?

If you downvote, please reply and explain your position. I simply asked if this Gmail PM is going to fix the problem with Gmail in the comments about that problem. So who is upset with my comment and WHY.

Potentially because you came off a bit aggressive, and because he already explained[0] that they had an analyst looking into this.

[0]: https://news.ycombinator.com/item?id=19759411

Apologies for unsolicited advice, but since you’re newish...

Downvotes don’t mean anyone’s upset, there’s a large variety of reasons people downvote. Same is true of upvotes, there are lots of different reasons for them. Try not to take either personally.

Downvotes don’t require explanation, just like upvotes don’t require explanation. It’s a great idea to be curious and seek ways to avoid downvotes, always good to assume you might be wrong. Less so to demand you were right, even if you are. :)

Also be aware that HN guidelines suggest avoiding complaining about votes you get, https://news.ycombinator.com/newsguidelines.html

Hope that helps, cheers!

While I know those guidelines, I still hate it when people get downvoted and there is not a single response (unless the comment obviously offends the guidelines). I mean, especially because people are voting for various reasons you can't even be sure that you broke the rules when you get downvoted and if the person has no clue what is wrong with his comment he doesn't know what to change.

So thank you for explaining the situation to craftinator :-)

Thank you for the advice! I've always seen HN as a great platform for thoughtful discussion. I don't see downvoting a comment that doesn't have any replies yet as a way to participate in that discussion. I was genuinely curious what view point a person could have to not find my comment beneficial to the discussion. I'll try to make these queries a bit less complaint-sounding in the future! Thanks again!

Great to hear the positive response! FWIW, initially on HN I felt pretty frustrated with what I called ‘drive-by downvotes’ with no explanation, and also just generally upset when I’d get downvoted. That’s partly because it didn’t happen that often, so it felt more serious when it did. And sometimes it feels unfair, for sure, especially when I don’t know why and it doesn’t seem warranted to me. But over the years, I’ve become more okay with the vagueness and uncertainty that comes with it. The votes are communicating something, so I try to assume I did something wrong and just maybe don’t get what it is yet. But you never know other people’s interpretations of your words, or their moods or mental states either, so sometimes I just have to let it roll off. It’s fascinating to see something I write get misinterpreted after I thought I was really clear. And I don’t see this happen that often, but because votes are used to rank the comments, sometimes people are just moving the interesting discussions up and the less interesting ones down, so there might even be nothing at all disagreeable with a comment, it’s just that there are other really good discussions nearby. Believe it or not, I actually don’t ever downvote because it bothered me to get them myself, I just ignore threads I don’t want to read and upvote threads that are interesting. I usually upvote people who reply to me as thanks for taking the time to read & reply to me, especially when they disagree with me, so that’s an example of using votes for something other than liking or agreeing or ranking. Anyway, HN does have some pretty thoughtful discussion even with the occasional downvotes, so just seek out the good stuff!

I believe people are frowning upon what seems like an open confrontation.

I have never used Postmaster-tools. Is there any particular reason why it requires a Google-Account?

a) thanks for the response b) do you have any clues why this failure is so sporadic? That is to say, why does an email from domain/server x get blocked on one day but get through a few days later (repeatedly)?

Why can't I even upload a simple text file to hangouts

What about amp for email as network effect?

Google does not have incentive to fix this, because they can simply not fix it, and get along just fine.

That's the point.

It doesn't have to be conscious dereliction, it can just be 'big, dumb bureaucracy'. Interestingly, operational ineffectiveness can happen within well run organizations, particularly if there's no poignant reasons to change.

These kinds of things tend to happen when one part of the organization is mining Oil or Gold and throwing vast surpluses around the rest of the camp.

You have $100 Billion dollars to fix the problem - a hoard of the best and most highly paid talent on planet earth.

So fix it.

I've written a similar article to this a few years back, so this isn't a new problem and obviously little has been done to fix it:

https://penguindreams.org/blog/how-google-and-microsoft-made...

I have SPF, DKIM, reverse DNS, DMARC .. all the things, and yet I still get e-mail dropped.

Will you guys fix Postmaster tools for small servers? What can I do to send e-mail to my friends?

If you deeply care what is the proper way of letting you know about it? I'm sorry but calling these relationships caring is rubbish. If Linus himself said that no one in the mail team listens, then what do the devs that are not the most popular name in the open source world have to do?!

Google could fix this rather simply. Have an opt-in system whereby anyone sending you email is charged a penny. (Of course, this will initially mean the sender will need to get a google account, but as this catches on more email providers will provide this service.) Split the revenue between the recipient and Google.

I don't mind spending a penny to send an email, I don't send enough to make any difference. But to spammers, it makes abusing the email system unprofitable. And if I receive half the revenue from people sending me email, it'll likely work out to costing me about nothing.

You have just reinvented postage stamp.

Exactly. Except the USPS doesn't share revenue with me.

Reply on news.ycombinator.com