Why did ProtonMail vanish from Google search results for months?
techcrunch.com
techcrunch.com
Two days later my product completely vanished from the Google rankings. No cause that I could think of; I wasn't doing any questionable SEO or other manipulation at the time. I later found out there were Google people at the conference, though I don't know whether they were at the talk or not.
Correlation != causation so I'm hesitant to definitively call shenanigans on this. But it does make me believe ProtonMail's story a little more.
You can't prove how a black box algo ranks you, but you can do a damn sight more than a throwaway comment accusing Google of cooking their algorithm out of spite.
(Disclaimer: I work for an Alphabet company)
However, they're the only ones that know their ranking algorithm. This means they can tailor their sites for the best possible ranking. It also means they can rewrite the algorithm in ways that better rewards Google (Let's give a boost to sites with the highest traffic, and exponentially drop off the boost as traffic decreases. Oh look, it turns out our own Google.com is quite popular, all of our services are under that domain, and they'll all see higher rankings).
I think you have a reasonable point, but given their prior experience with publicly criticizing Google, they also have a reasonable desire to remain anonymous.
I mean if Google is going to penalise them and risk their search integrity for a quip what's to stop them asking a bud over at HN to see if this account's IP is associated with another?
Infeasible of course but around the same likelyhood of happening
I disagree. It's much easier to identify me by my real name than by asking a third party to analyze server logs to find an IP and match it to users (many of which may be using the same public IP).
Those are the two things I was comparing, not how easy it is to identify someone.
I'm not comfortable with saying any more because, as hackuser says, I'm not risking the last two years of work (we've finally got back up to that level of Google traffic, though fortunately other traffic has grown immensely). I can simply offer this up and let people disregard it to the extent that they're willing to trust a throwaway account. You don't trust it at all, and that's fine.
Never blame on stupidity what can be blamed on malice when power centres are involved.
http://www.ted.com/talks/andy_yen_think_your_email_s_private...
Disclaimer: This is speculation. Take both this article and my post with a grain of salt.
Only if you do it wrong. If you do it write, there should be negligible impact.
Moving domains always comes with its risks.
However, it doesn't change that I use Gmail and will continue to use it. It's still the better product for the money (free).
Not sure how I missed it.
PS: Same goes for http to https, some years ago you've got a huge huge drop in rankings while today it's a no brainer.
Moving a domain is always a high risk operation for your search results position.
Unfortunately a lot of ecommerce sites are of low quality and coded by the finance directors golfing mates "friends" company.
We see so many different cases here on hacker news that have one thing in common: Bad things happen and those affected have no idea why these things have happened.
It doesn't mean that companies need huge support organisations. I'm sure 99.9% of these questions can be answered by an automated system. Support can dig into the more exceptional cases and doesn't necessarily have to be free.
If people wanted transparency they'd switch to companies which are transparent.
(</NaiveToMakeAPoint>)
In other words, the people who need more transparency aren't the ones who can choose what search engine is being used.
But more generally I see it like this: Without the law, Google would not exist as a legal entity. Incorporation is what allows Google to become a legal entity and it is also what protects Google's owners from having their personal possessions seized in case of insolvency.
So Google's owners have asked society for protection beyond what is provided solely based on market principles. In other words, they wanted rights, not just incentives.
I'm asking for the same thing. Rights that are independent of my ability to pay and are difficult to emulate through market mechanisms.
I think corporations are part of market principles.
Your analogy is incorrect because the rights are different.
I am not you have any idea of how long it takes for General populace to realize that they are being screwed.
Ordinary people don't yet know because they trust Google - A trust which was build by constant "Don't be evil" rhetoric for over a decade. This trust also helped them acquire a monopolistic position so when they break that promise they need to be investigated.
Search is now a public utility and Google should be treated as utility provider.
And those are just the manual changes. There are fully automated systems that crawl the web and use information from millions of web pages simply to rank your website. Any of those web pages changes, and your ranking might change with it. For true transparency, you should be able to verify that you got the right ranking as documented, yet merely calculating that is going to require you to get a copy of every other webpage in the world. No simple task.
Someone at Google, hopefully. At least I wouldn't want to work at/be customer of a company where no one knows how their core product actually works.
If Google employees are able to keep track, so are regulators. If not, then the "cost" of requiring it may actually be a benefit.
There is a more general issue here. As we use more machine learning and AI, the problem of having to explain why a particular decision was taken will come up more often, especially if the decision has grave consequences.
All "AI-first" companies would be well advised to work on this problem. This is like computer security in the 1990s. It's going to be absolutely central to many applications of AI and it will be a key legal issue in future.
[1] http://www.usatoday.com/story/tech/2015/07/01/google-apologi...
In search, email, etc there's continual competition between people trying to trick the span detection algorithm and people trying to extend the algorithm to counter them. Secrecy is a critical tool on both sides of this fight. Force transparency on the algorithm maintainers and it's output will get much worse.
Keeping the algorithm secret has downsides, but is worth it (for society at large) on balance.
(Disclosure: I work for Google, on unrelated things, and don't know anything Google-specific here.)
But if there is a dramatic change in ranking or my site is removed entirely, it should be possible to ask Google to investigate (possibly automated) and tell me if something out of the ordinary has happened.
Secrecy shouldn't mean to hide simple mistakes on either side. That makes search results worse not better, as it was clearly the case with protonmail.com
Why is it that large companies like Apple and Google are so hard to reach? Remember the story of the Dash iOS app developer that got kicked out of the store. He also practically was unable to reach anyone.
http://www.loopinsight.com/2016/10/10/apple-responds-to-dash...
He's not in the App Store today, because just posting that call without consent shows he can't be trusted.
They may have tried to contact him at the address of the second account, which was spamming the store and may or may not have been him.
But instead of accepting this resolution allowing everyone to safe face he seemed to be drunk with the power of the solidarity he had gotten initially, or he just wanted to really win the argument. I don't know the motivation, but anyone could have told him that trying to blackmail Apple was not going to end well,
So you’d also say someone should never sue Apple for doing something bad, because they might abuse their monopoly power against you if you try to win?
Paying user of dash here and I've been very impressed by how the developer has allowed the Windows and Linux app zeal to use his document sets and format for free. Only condition was that they don't release a competing version for OS X.
I think he deserves the benefit of the doubt.
Another example is the recent Spotify bug, which did not receive any attention until the story made it to the top of HN. I think companies actually cause damages to themselves by not reacting sooner.
I find hundreds or thousands of complaints. Many are legitimate bugs, and some are user stupidity ("I need to check my mums email but she won't give me her password, can you let me in anyway").
For those which are legitimate bugs, I know, through user metrics, that they only affect a tiny fraction <0.1% of the users. I, and the rest of the engineering team have to prioritize our team between fixing these bugs for the 0.1%, and making the product better for the 99.9%.
It can be a tough choice, but when you've decided that it's time to move on and develop new features, hearing about individual instances of rare bugs is no longer useful. We just aggregate how many users are impacted by each significant bug, and from time to time quash the top ones.
Manually editing a database entry for a single user is no longer privacy-justifiable, so basically the only fix we can do is to fix the bug for all users at once. And if the bug is only affecting 15 out of 300,000,000 users on a product that makes 1 cent per user per year, I can't afford to spend more than 30 seconds on it really, yet most bugfixes are at least a days work.
Steve Jobs seemed to have an ear or the 0.1% :) I am pretty certain one (rare) OSX bug got fixed because I emailed him.
However, I think this approach should be criticized. It's a very concrete improvement for the 0.1% of users dropped in favor of a very vague improvement for an unknown subset of the 99%. Also, the bugs have severe consequences for the users (as in the OP or the Dash case) it would be downright irresponsible to be customer of a company that acts like this.
It's like a DC politician who's above the law - you have so much power that you can literally do what you want.
The solution? Stop using products of the biggest players and start looking for alternatives.
They don't support users because of arrogance. There is no viable alternative, they know that. And even if it costs them more revenue than the total costs of a support team, Google's culture is so accustomed to lavish spending that even 1B in lost revenue is just a blip on the radar.
If your account is worth more than a few cents, they'll gladly speak to you.
KLM doesn't have flights for 3 cents, so doesn't have such a support-timewaster problem.
An account holder chosen at random is most likely to be in the former category, unlike klm where a customer chosen at random is likely to have a much higher margin worthy of a few dollars of support costs.
Some might argue a company should support a customer well even if doing so puts that customer as a lifetime lossmaker though.
The same way Amazon does? How about banks, airlines and other similar.
I understand that there's little motivation for some online services to improve their customer service experience but Google is especially bad. Even when making a purchase from them (an area where both Apple and Microsoft do much better), you're dealing with a bot with absolutely no other contact point.
And I personally think that’s actually good. Open all the antitrust cases against Google at once, so it can be broken up.
I have had a ProtonMail account for a few years and the system keeps getting better. The one feature lacking, and keeping me from using it as a primary email service, is not being able to backup my emails locally. I think that is on their TODO list.
I am referencing the official ProtonMail blog post (https://protonmail.com/blog/search-risk-google/) linked by another commenter in this thread.
I also want to say I'm not a Google employee and have no vested interest in the authenticity in their algorithm.
It is very fishy that according to PM they contacted a Google rep via twitter and got a "we fixed something" response from them.
As an SEO person, my opinion is this:
As an encrypted email service ProtonMail probably gets a lot of foreign and otherwise "undesirable" (or deplorable ;) ) links in the eyes of Google. Google lately has been doing a lot of algorithmic changes to actively sift these types of links out of consideration when ranking sites or dampening them severely.
My guess is that Google was basically penalizing PM for the links they were getting without notifying them via Webmaster Tools. Whether or not there was a manual penalty involved that they weren't notified about also adds a little bit of shadiness. Maybe Google thought PM was trying to game the algorithm or at the very least many of the links they were getting were of a spammy nature and they ran a penalty on them.
The redirect from them changing a domain in the past can also compound issues with Google. Sometimes they consider redirects spammy depending on what kind of redirect is in place.
Without more info it does seem weird that they would "fix" something not only without warning them in the first place but not explaining it afterwards? Most web admins have to use what's called a "disavow" list to remove the penalized links but apparently their rep seemed to do this for them. Probably due to the anti-trusty viral nature of the complaint?
I don't know what SERP tracker PM is using for these stats. Barring there were no errors in that system, they can trace the lack of rankings to a lack of organic traffic, and they were given no reasoning to the penalty or what was going on...
They may want to hire a lawyer and start an anti-trust suit on Google. This is very fishy without any official information from Google.
https://protonmail.com/blog/swiss-surveillance-law-referendu...
In the blog post, proton says they aren't impacted by the new law. Do any third party analyses agree?
Google DuckDuckGo Yahoo
Proton Mail 1 16 -
Tutanota 7 20 -
Gmail - 10 1 (G Suite)
We don't even rate a mention for "professional email" because it's all about how to not come across like a n00b when writing emails.
(interestingly, bing search for professional email has an ad for Google's email above the search results)
- Google, an advertising company, benefits from having access to the plaintext of people's messages. They can advertise this as 'secure' but only for the comm channel. - Proton mail, advertises itself as: "ProtonMail is the world's largest secure email service, developed by CERN and MIT scientists. We are open source and protected by Swiss privacy law.", featuring end to end encryption, anonymous email, mobile clients etc. - Fastmail: "Secure, reliable email hosting for businesses, families and professionals. Premium email with no ads, excellent spam protection and rapid personal support."
If I'm a product manager of GMail, I'd be more scared from Proton mail than FastMail. In fact, anything featuring end-to-end encryption will be approached with negativity, because then GMail can't target ads, which are the main cash cow for google.
It doesn't have any additional security benefits beyond regular webmail providers that provide TLS:
- Encryption is performed inside the browser. By delivering a modified JavaScript payload, it can steal encryption keys.
- No support for POP, IMAP, or SMTP. If you want to use GPG, you can't.
To see it being compared to Fastmail and Gmail, which don't engage in parlor tricks, is a travesty.
Sure, you have to trust their software, but is that fundamentally different than trusting the GPG software, or did you do a full audit of that?
GPG has a track record of being secure against many adversaries, including the NSA. It's used by most Linux distributions for package signing, so you probably already depend on it -- even if indirectly.
I don't expect GPG to be completely secure. But it's not based on deception.
I get that there's a difference in degree, i. e. GPG binaries being checked against hashes and having a long track record as an organization, but is that fundamentally different?
I could see ProtonMail evolving to, for example, using a browser extension that allows you to use a known-good version of the crypto library, and informing you of changes.
Point being: it isn't perfect and I'd prefer something based on standards. But e-mail encryption has failed, even though it is often more personal than websites where TLS has been successful. ProtonMail is a legitimate attempt in a space that seems to need a new approach.
GPG is used asynchronously. In many cases, everyone would need to be compromised to go after one person. That raises the chance of discovery.
A browser extension is a fantastically better idea. Google previously prototyped most of the work for this: https://github.com/google/end-to-end
Further to that, I'd trust Google's infrastructure to withstand compromise much more than I'd trust a datacenter run by a small company that I know much less about. (Tinfoil hat: sure, I have to assume that the NSA has a copy of my Gmail inbox, but god knows who else may have owned ProtonMail.)
How often do you download ProtonMail's Javascript code? Several times per second, as long as you're using it.
I chose Fastmail. I pay for my email service and don't expect them to sell me out. Get to use a functional email system that lets me search my emails(ProtonMail lacks this ability).[0]
Being aware that my emails could be snooped on by some government somewhere.. so nothing sensitive goes in there. That's what OTR or other personal encryption systems are for. [0] - https://protonmail.com/support/knowledge-base/search/
Google probably dampened the authority of many of the sites linking to ProtonMail, for one reason or another.
This most likely brought Protons rank down without them doing anything wrong.
SEO it is hard to say. They want to compete on "email" and "secure email" keywords it is not going be easy.
G Suite Email is a paid service that doesn't sell your privacy, doesn't serve ads, and has extremely good service.
G Suite is an american service and will give away your privacy without a proper/open court case when receiving a gag order. This is very much in contrary to PM.
ProtonMail on the other hand, while not at the moment a threat to gmail does do a lot of brand damage to it, as it highlights the privacy problems people have with it.
ProtonMail said they weren't ranking for "encrypted email" and "secure email" [1] for a year, not just "email". How many "much more popular" than ProtonMail "encrypted email" services do you know?
And no, using StartTLS doesn't count. I'm talking about email services that have promoted themselves and in the media as encrypted or end-to-end encrypted email services, and would therefore get the ranking for those search queries.
ProtonMail wasn't talking about ranking high for the word "email". Obviously that would be quite difficult to impossible for a year old company.
Assuming the blog isn't blatantly lying about their position being lower just recently, it's proven that google did penalize them for a while.
As to the reasons, I tend to assume bugs or coincidences. I just can't see Google doing something like that on purpose, when they'd gain a minor edge in the webmail business at a huge risk to their reputation, a smoking gun for all those regulators poised to hit them with charges of anticompetitive behavior at the first opportunity.
The fact that they went through Matt Cutts to get this resolved is telling - since he's the public face of Google for anti-web-spam/anti-blackhat-seo.
> @mattcutts We know Google is intentionally hiding ProtonMail from search results. Interested in talking before our data goes public?
Apparently threats help get you attention.
He even left google and works for the US Digital Service since this summer.
And on the topic: Nope, no penalty. At least no ordinary / known version of typical penalization.
"One example from the FTC report shows how hazy the line can be. When testing changes to its ranking algorithm, Google has “raters” manually assess different sets of search results for the same query and rate which one is better. In one such experiment, Google tried demoting “comparison shopping websites” like NexTag to see if its raters liked the results better. When they didn’t, Google tweaked the demotion algorithm until they did."[1]
[1]http://www.slate.com/articles/technology/technology/2015/03/...
However, in general, I am not a fan of regulation. I wonder if this problem could not be better fixed by an open source search engine "assistant" that simply polls several existing (competing) search engines? Perhaps implemented as a browser plugin?
I base this suggestion on the assumption that not all the usual search engine providers will have the same conflicts of interest. More specifically, in this case, they are not all email providers themselves. For the same reason I trust more the search engine providers who have not added to it all kinds of other business interests.
Any thoughts on that? Has anyone tried it?
[1]: https://en.wikipedia.org/wiki/List_of_search_engines#Metasea... [2]: https://github.com/asciimoo/searx
Does anybody have any recommendations for alternatives to Google Drive?
http://www.cnbc.com/2016/07/19/google-could-face-3-separate-...
To expect otherwise is foolish.
Yes, they are, if they want to access the market at all, they have to provide results to everyone, and can’t discriminate.
> And they should be expected to market their own products.
No. They have > 90% of the market, which is used by > 50% of the population, meaning they’re automatically a public utility, and can not provide any positive or negative discrimination to their own or other services.
Any treatment has to be completely fair and equal, and if their own services can get integration, so any competitors have to be able to do. (For a fair price, of course).
Is that your assertion or is that through legislation where you are (which is where?)? Thanks.
Even without, it just takes to open a throw away account here, and say something.
So it should be secure enough but for sure, the implementation is still not perfect at all, especially if you want to communicate with non-protonmail users.
I remember I was able to get key passed through the network in the not encrypted form, I was looking into the network tab of the browser's dev console. When you do login look at the response of the https://mail.protonmail.com/api/auth POST query.
Instead of publicly looking for a scapegoat for their mistake they should tell readers of their blog the one lesson to be learned from this story: Never use a country TLD for something targeting more then this country!
I must say I don't think they even have customer support. I emailed them and opened tickets multitude times with serious questions (looking into moving entire enterprise with 100+ email addresses) but I guess I was neither their type of client nor they simply don't have customer support at all!
This issue turned me off and against them entirely; sorry.
What they are doing, both from the backend side and the multiple clients side, is a lot. Chances are they are just really busy with the number of employees they have.
There are ways to reach out to google and get an answer
So in Protonmail, for mails inside the service, from protonmail user to protonmail user, they afaik use a pgp implementation and handle the key exchange.
To send secure messages from protonmail to another service, you can encrypt the message / conversation with a password and the recipient gets an email with an url, where he can, given he has the password, decrypt the message and reply securely.
Or Google might be broken up – there’s also antitrust cases against them in the US.
With more than half of the population using them, and them holding > 90% in multiple such markets, they fall into the category of monopoly that automatically becomes a public utility – they’re supposed to be just a dumb pipe.
Defusing a court case would seem helpful. Explaining to those worried by this might be good.
Acknowledging you screwed up, fixing it and publicly stating you will do better (then doing better) can actually boost the way a company is seen.
IANAL but I doubt they'd get very far. What would their actual argument even be? Google isn't obligated to include every web site in existence in their index.
I believe in this case protonmail was hit by an algorithm based derank (not a manual action). What causes it is anyone's guess. But something series of events hurt their ranking. Its also important to note that PM was NOT delisted. They still were getting traffic from Google, they were just deranked.
No, but to have a competing product appear highly ranked at first, then drop off their search engine entirely and not return until there is a public outcry, screams of anti-competitive action on Google's part[1]. That said, I don't think a lawsuit would get very far either. I'm just saying the whole affair is fishy.
[1] Or, if you want to go tin-foil-hat with it, perhaps a certain government told Google it didn't want people to know about a secure email startup that resides in a country that is known for upholding privacy rights. I don't buy that one myself but I've seen it bandied about in the past in similar cases.
See these screenshots from ProtonMail:
https://protonmail.com/images/slider/compose.jpg
So, legally, this is already nothing.
Second, this is not something Google invented – Google started with webmail, but many other companies had designed stuff like the in-frame editor, or other concepts before Google.
So, even if ProtonMail outright copied Google, Google, because they copied together many other people’s ideas, wouldn’t have reached the Threshold of Originality.
https://techcrunch.com/2015/08/03/swipe-to-patent-design-pat...
Even if nothing applied in Switzerland, the countries supporting such things might block their local presence there under grounds of violations. This risk actually is why some companies, esp Chinese, won't do business in the U.S.. If this played out, they'd be blocked from serving customers in U.S. or any nation that upholded the claim against them.
You might not like it, but it is generally legal to create products that clone others.
edit: spelling