Google Has Most of My Email Because It Has All of Yours
mako.cc
mako.cc
GA is the standard analytics for a huge % of websites - so even if a website doesn't use GA for tracking traffic, Google still has the referral data. And things like Google Adsense (i'm pretty sure that sends back the referral data too, for tracking click fraud).
There is no way really to avoid Google knowing a lot about you/your website anymore.
The email problem sucks though. We need end-to-end encryption, on all mail, today.
Btw. I've been also running my own mail servers for ages and have been fed up with Gmail users.
Google Analytics is on a substantial proportion of the Internet. 65% of the top 10k sites, 63.9% of the top 100k, and 50.5% of the top million[1]. My own results from a research project I did using the Common Crawl[2] corpus estimates approximately 39.7% of the 535 million pages processed so far have GA on them.
The real key to tracking is the referrer data. For the vast majority of clicks, you land on a site that has Google Analytics or you've just left one that did. As Google Analytics tracks your referrer, that means they still have your full browsing history if you jump from GA => !GA => GA => !GA => ...
According to my research[3], Google gets activity information on 51.43% of the 42 billion links analyzed in the 535 million page corpus as either the start or end of the link uses Google Analytics. This activity means they can accurately track browsing history on most sites, even those that don't use GA, simply as timing information, referrers, and knowledge of the web graph end up leaking user activity.
Used in an anonymized fashion, this is beneficial as it helps Google understand real world web traffic and hence rank search results accordingly (far better than simulated activity based upon PageRank or similar). In the theoretical situation you drop anonymization is where this gets troublesome.
If you're interested, there are more details at "Measuring the impact of Google Analytics"[3], though much of the discussion is on Hadoop + Common Crawl. For a privacy focused write-up (primarily worried about the NSA using Google Analytics), refer to "Google, make Google Analytics HTTPS by default"[4].
P.S. Everyone who notes "Google Analytics is easy to evade" are correct but missing the broader point -- the majority of web users will never do that.
[1]: http://trends.builtwith.com/analytics/Google-Analytics
[3]: http://smerity.com/cs205_ga/
[4]: http://smerity.com/articles/2013/google_analytics_and_nsa.ht...
In the unlikely event that fake activity became a problem, Google's well equipped to deal with it. They have a great deal of tech and brains in place to detect fake ad click activity, which is vaguely related.
Here's something even more troubling. Take an 'anonymous' crime reporting site[1] and put Google Analytics on it. Put it on every single page, even on the page with the forms to submit anonymously. Not bad enough? How about a similar site, only this one aimed at reporting corruption[2] and try the same thing. What could possibly go wrong?
Both sites are well aware of the issue and have written me back when I pointed this out. This level of trust in an American ad company is curious. All I can do now is hope that whistle-blowers wanting to report corruption are savvy enough to avoid the web forms.
Imagine you are a government worker somewhere and you see evidence of corruption and report it. From the same machine you signed in to GMail with. Now consider that your local government can order Google to secretly hand over tracking data and forbid them from notifying the crime reporting site(s).
[1]: https://crimestoppers-uk.org/give-information/give-informati... [2]: https://forms.theiline.co.uk/integrityline
This is an important point here because it's in stark contrast with Google e-mail. Google's e-mail servers are exceptionally difficult to evade.
I am (apparently) able to evade Google's attempts to track my web browsing habits with Google Analytics. I can take responsiblity for that myself.
This is a technically difficult opt-out process is IMHO against the spirit of "don't be evil", but at least there is an option.
On the other hand, I am totally unable to prevent Google from building an accurate profile of my e-mail habits. The only way I can opt-out here is by significantly curtailing my e-mail habits (or by insisting on PGP, which will have the same effect). This is significantly more frightening to me - I have no choice.
The amount of tracking google, facebook does is insane. And I hate the fact that none of my non-techy friends even understand it.
Next time you link to a .css with those wonderful free Google Fonts, ask yourself - what's in it for Google?
Then take a look at all those ajax.googleapis.com links pulling down jQuery libraries and wonder the same.
I worked there for years. Seeing really deep, well thought out business plans there was a rarity especially for small projects like hosting web fonts or running DNS resolvers. Heck, even for very large projects sometimes the accounting was unbelievably carefree.
Pfft.. What was I thinking? It's the original Dont-Be-Evil company, right? Of course they are giving away tons of freebies just because they are awesome. They just run a money printing press for an extra minute and those huge budgets will materialize out of thin air. Yay.
Or, just, you know, disable javascript entirely.
Except it's not because the problem's still there for everyone else. Do you really that being shielded yourself but allowing your non-techy friends and family to be tracked is a 'solved' problem?
I always use Piwik [0] - it's excellent, open-source and most of all: it respects your users' privacy by not letting any third party like Google track them.
My advice: Use it too and let your users know that you do so because you respect them.
[0] http://piwik.org
If you use Google Talk, every conversation you've ever had will be recorded and indexed and tied back to you. If you use gmail, same deal. Even your drafts of unsent emails will be. If you use AIM, same deal: every conversation you've ever had on it will certainly be logged somewhere and tied back to you. Yada yada, same deal for almost every chat program, because almost every chat program has no clientside encryption. If it does, it's not very popular, or it's hard enough to use to where people will think you're paranoid if you ask them to go out of their way to "download this chat program that lets us talk without anyone logging it."
I think the endgame here is to watch what you say. It's safest to assume every text conversation is public. How many of us have said something in text to our families or friends that we'd be extremely uncomfortable saying publicly? It's a little unsettling.
Then again, hopefully when the TextSecure people ship their browser-based chat program things will improve somewhat, because you'll be able to talk to someone else without the conversation being duly noted. (There will probably still be metadata that ties you to the fact that you're talking to someone, but at least the content will be protected.) Hopefully it will be easy to use... I wonder if they need any help in that capacity.
Sadly, people who have been saying this for more than a decade were derided as paranoid.
So it seems important to come up with a technological solution to the problem of how to communicate without all of it being logged. It's a difficult problem because it's hard to get other people to actually use whatever you come up with. That's why I'm crossing my fingers that TextSecure's browser plugin will take off, because if it's as easy to use as email and as powerful as email, it could have a very tiny chance of becoming the next popular communications platform. At that point no one would have to trust any company to preserve privacy, which seems valuable.
EDIT: I'm confused why my comments were moved to the bottom of this thread, because they don't seem offtopic. For example, the second topmost comment is also about encryption: https://news.ycombinator.com/item?id=7731216
That's patently untrue. What are you basing that on besides your own paranoia?
Would it? The NSA is reading everything and I haven't seen see many changes. I think we as a society need to sit down and figure out if these services should be in the private domain or instead implemented in the public domain that matches with our values. We spend billions, why can't we spend a few to make an internet that doesn't spy on us.
I don't think I could stop using Google. I don't know what would make me stop.
Where is this public domain that you trust so much and why are they more honest that Google?
You piqued my interest, what would happen if they killed somebody? Probably whoever asked for it would get in trouble (unless they deleted the logs...), but I doubt I would or could stop using Google. We use web-search like a utility.
>>Where is this public domain that you trust so much
If it works or not is another story, but we have a de-jure mechanism for government accountability wherein nothing exists for corporations. We might be able to make a publicly accountable internet infrastructure, but there is no similar mandate for a corporation.
What? No.
They saw the PR pouncing they were taking, so they hired a former judge to rubber stamp such things in the future, and had the gall to call it a "judicial process" (even though the guy is just a microsoft employee at this point).
Considering the usual public reaction to that kind of thing I'm surprised it doesn't get mentioned more often.
>I've always wondered whether Google ever digs into communications in a situation where they're trying to decide whether to acquire a company. It seems like reading a company's email would be a reliable source of information about whether they're on a genuine trajectory or whether e.g. they're having trouble with their investors.
Now there are a range of ways that could happen, from Larry Page looking up the emails personally, to some middle manager involved in the deal getting their friend on the gmail team to unofficially take a peak to look for some specific thing. Things happening inside a company can happen in all sorts of ways that aren't official executive decisions.
Individuals motivations do not even have to be aligned with Google's interests[1]. Maybe they want to be able to better position themselves if the deal goes down the tubes, or get information about any employees at take over targets that might be going to replace them in their role. Internal politics at large corporations is endless. From the outsiders point of view the actual mechanism doesn't matter too much.
Also I think the situation where a sexual predator was attempting to coerce minors into sex is (or should be) a much more serious public scandal than an executive decision to look at the emails of a startup most people have never heard of and don't care about would be. I imagine many people would blame the startup for being foolish enough to use gmail and Google would deny it and it would all blow over in a few days.
Go upstream to the Snowden 'allegations' and apply that idea to Lockheed Martin. They can get every government contract they want or whatever war they want because they know what their rivals bids are (because they built the NSA). Plus they run all of the computer security in D.C. and there is no way that any communication in 'elected government' misses their eyes.
Even sloppy bidding processes in random third world countries include the concept of sealed bids - noone in government should have any info on any bid amounts before the bidding is closed; if Lockheed Martin wants to know my bid before making theirs, then they'd have to wiretap me.
Of course, there are many other options for fraud and espionage, but getting rival bid amounts before due time shouldn't be one of them.
Basically instead of an actual email the recipient would get a link to an https'd page on my mail server and a brief note explaining that due to delivery policy the message is available only at the link.
The reason why I started looking at this was that I was buying a house and the broker person was using gmail to handle the transaction. From negotiation to all the forms with all juicy details. I switched him back to the fax mode, but it got me thinking that it'd be nice to have a system in place that would try and offset such negligence, automatically.
I never got past a rough prototype though, but perhaps I should've.
Not foolproof or without problems (there are not only crawlers and bots at these companies, there are also people who receive email), it would solve the problem in most cases.
A general purpose email version of this (both with and without self destruct (which should have options such as timer, N views etc)) would be awesome, especially if it automatically intercepted emails and moved them there. Would have to be something I can host myself, of course.
It's bulk surveillance of everyone, everywhere and at all times, that's a problem.
Taking measures which raise the cost of surveillance helps ensure that if they get your data, it's because they really want it. Not just because they can.
So if the default was EoE, it still wouldn't matter.
Global EoE encryption's benefit is making it significantly (and perhaps prohibitively) more expensive to engage in bulk surveillance.
Really the only readily-practical approach for NSA-proofing will be something that's user-friendly in the UI and encrypts end to end. This prevents snooping in transit, but still exposes metadata, and of course there are endpoint attacks.
Peter reminds me of the old "If you have nothing to hide..." fallacy. I'd have expected more from the EFF.
Yes, anything really sensitive should be PGP'd anyway, but using gmail still gives google the opportunity to do analytics's.
Peter seems to believe that using Gmail makes his friends' privacy incrementally worse, and yet he is contributing to this problem.
What he really means is: Gmail is more convenient than the other options - there is no better solution to this. And, that's the problem.
So, to answer your question more directly, facebook doesn't "get a pass". Facebook simply doesn't get used.
Similar to the problem with Google having access to a large percentage of emails, Facebook will have pictures of me regardless of whether I choose to personally have a Facebook account or not. Assuming that I do have a Facebook, with a network of friends, they can also tag me in their pictures (and therefore available to Facebook) even if my own privacy settings are turned all the way up.
To most people, it's black magic. In fact, they don't even know that such tracking is possible.
Facebook has your pictures when your friends that are on Facebook post pictures of you.
And even if you are not on Facebook,i'm pretty sure facebook has a "shadow account" system to track people even if they dont signup.
The "People You May Know" screen on that Facebook account has plenty of people I do in fact know.
I imagine through people uploading their address books and then Facebook mining shared connections, they inferred a bunch of my network without me doing anything at all.
That said, I wonder if meshnet protocol could be utilized as an alternative. Although the traditional mesh network is impractical at scale, a virtual version, or an email-serving proxy network of some sort, could be beneficial.
Well, beneficial if you'd consider keeping email off Google's centralized servers a good thing.
Today it can be (and often is) frustrating even for established companies to get their mail delivered to all sources. I've had repeated frustrations especially with Yahoo, but also AOL (both continue to have a large number of addresses, if not active accounts -- problems in scrubbing old email addresses is another challenge). Larger companies may have their own idiosyncrasies regarding accepting email -- even with SPF and DKIM records, I've not infrequently encountered companies (some of which, granted, do things involved making littler things out of little things called atoms) who requested (and presumably require) the specific IP address of our outbound mailservers for communications.
More generally, email badly wants to have some sort of reputation layer put on top of it, though how to accomplish this has eluded general solution (SPF and DKIM are only band-aids, and already break a lot of legacy behavior). Total encryption would be good, including of headers. It's a bit of a mess.
Source: I was part of the Gmail spam/abuse team for several years.
Any given user, and often large groups of users (a company or organization) are going to have traffic patterns which strongly favor a small number of other hubs (mailservers), in general. That's going to be, generally, high-reputation and high-value traffic. You want to ensure that it gets through. That solves most of your problem right there.
Some of those sources are also spammers or low-value -- email marketing and the like.
Everything else is, well, everything else. Might be spam, might not. But as a first pass it tends to be less valuable. Which means you've got an immediate and low-cost option: deny first delivery on a nonpermanent basis.
If it's a well-behaved system, the delivery system will-retry the transmission in about 4 minutes. If it's a spammer, odds are that it will simply bail on delivery, or fail to honor the usual retry fall-back schedule. In the first case, problem solved, in the second, you've now got an additional datapoint for the source: it fails to adhere to conventions.
All of this is happening largely at the host-to-host level, not individual senders, so that you're both getting a large level of aggregation (a new user or service transmitting through a known host isn't a blank slate, you've already got a delivery history), and the overhead is smaller.
Yes, there are also reputation and other systems (IronPort / Senderbase, now part of Cisco, for example, as well as the DNSBLs), many of which are accessible via DNS queries, though the cost of those queries for a busy system is itself considerable (you probably want to cache results, fortunately, DNS allows for that).
And all of that logic can be rolled up pretty readily within an MTA. That's one of the powers of free software: aggregating brains and experience.
I always ask myself, who cares? Worst case scenario, Google will sell this data to a government and I'll go to jail. The effort required to secure email at this point isn't worth the time or effort it'd take to maintain.
When I first set up my mail server I was pretty excited about being able to help everyone I know get their email away from Google, Hotmail etc. But once I had it running, I quickly realized that I didn't really want to give to very many people. Even if I trust all of my friends not to abuse, I cannot trust all of their computers.
In any GPG/PGP solution that I'd use, the encryption would be automated and transparent. In practice, the web-mail-client would anyway decrypt, store and index that email for convenience - no matter what you do, if your recipients/senders use some 3rd party email service, that email service would have access to your emails after the GPG/PGP layer is removed.
What GPG/PGP achieve is defense against MITM/phishing impersonation and secrecy while in transit between email providers; what it doesn't neccessarily achieve is secrecy in storage and defense from your e-mail client software developer. Coincidentally, these are the exact same security characteristics that a gmail user mailing another gmail user has - there are no third parties in transit; gmail can prevent insertions in the middle with a faked sender; but the stored emails are vulnerable to google itself and legal requests made to them.