You can’t buy Congress’s web history – that's not how any of this works
theverge.com
theverge.com
In the AOL anonimized data leak there were plenty of individuals identified:
http://www.nytimes.com/2006/08/09/technology/09aol.html
MIT researchers also showed that four anonymous purchases are enough metadata to identify 90% of individuals:
https://www.technologyreview.com/s/536501/data-sets-not-so-a...
And, a personal favorite of mine where researchers from Standford and Princeton are reporting at the World Wide Web Conference this April: "Researchers found that they could identify the person behind an 'anonymized' data set 70% of the time just by comparing their browsing data to [often public] social media activities"
https://www.techdirt.com/articles/20170123/08125136548/one-m...
It would not be hard to buy a zipcode worth of data and compare it to known facts about a person until you de-anonymized it.
There is a technique of intentionally anonymizing data [1] that I learned about because that Apple was talking it up in relation to storing health information. I'm only a layman, but my understanding is that it makes it much, much harder to do an analysis like you describe.
[1] The term is Differential Privacy. Here's a tutorial video (1 hour, 34 mins): https://youtu.be/ekIL65D0R3o
US Zip codes serve around 7,500 per code on average (US Census 2010) which is different than common wealth postal codes like in Canada where they serve an average of 19 households (25 - 75 people).
But, zip data can be interchanged with plenty of other unique identifiers on the web. Maybe it is browser language setting, or version of java etc.
Think of it like an Excel spread sheet, if in column A you can have options "1" or "2" then in a list of 100 people there will be at least 50 who share the same data footprint. If you keep adding columns from B onward with the same logic eventually you'll have pretty unique strings.
Things like searching history or web history are even worse. Ever done a search for a pizza place near your address, or Google map directions from your home to another location? That identifies you pretty easily. So does connecting to your works website, and the school your kid attends. Web browsing data is nearly impossible to anonymize by its nature unless it was compiled to something like "XX% of users in Zip XXXXXX visited website.com"
As for differential privacy, it is a nice emerging theory, but there are challenges with it as there is a significant trade off right now in terms of data accuracy when applying differential privacy. It is primarily effective at casting doubt on if variable "A" about user "B" in a data set is true or not, but if you don't have a specific target or specific metric then enough of the data is true that it could still in theory be deanonymized, and since most of the anonymity is based on incorrect variables in a data-set, all it would take to reverse engineer it is a large enough data-set and a few known variables.
I hope people like Apple continue to champion the advancement of differential privacy though - it is a major step in the right direction. But, being able to buy browsing history, even in aggregate does not protect individuals.
Forgive me for asking, but you seem to have two definitions of "anonymized":
- anonymized - not anonymized, but claimed to be anonymized
I think this argument (which I agree with) would be more forceful if we could stop calling non-anonymized data "anonymized". "Depersonalized", perhaps?
Then there is actually 'anonymized' data which would be the release of data in which you cannot in anyway identify a user. An example that comes to mind for me is the census releasing aggregate stats such as "14% of American's speak language X."
If the census instead had records of each American line by line, listing which language they speak and other associated factors about them then this data is likely to paint a unique picture of the individual even if their personal information like name and address were removed.
I think most data is very hard, if not impossible, to truly anonymize. Even if the search history that gets sold wasn't broken out into history per tuple/record, then you could still identify at least a few trends in it.
Does that make more sense? But yes, I agree that these companies are more attempting to 'de-personalize' data for the sake of research, but, that is far from anonymous and naming it as such is misleading to the public.
[...]it requires a social media feed that includes a number of links to outside sites. However, they said that "given a history with 30 links originating from Twitter, we can deduce the corresponding Twitter profile more than 50 percent of the time."
I'm paranoid enough to stay away from big social media altogether, but I realize that is uncommon.
I have very few tweets < 200, but I think you can find 30 outbound links in my first 50 posts since most of my use for it was sharing interesting articles.
Can you imagine combining that with something else as simple as which Oauth apps someone has approved in Twitter? You'd reach near 100% accuracy in no time.
hahaha. THat's quite a talking point. What would these "Robust safeguards" be? The history of pretty much every study on this, ever, is that it's pretty easy to deaggregate and deanonymize data.
But look i'm sure, verizon, at&t, etc, those great bastions of "doing it right", have done this right too!
20 bucks says if someone buys large amounts of the aggregated data, they can extract significant information that can be pinpointed to individual congresspeople.
Say it is indeed possible to pinpoint individual congresspeople, as the grandparent suggested. If some of these congresspeople have a browsing history they'd rather not have made public, identifying them could leave them open to blackmail.
Deny the claim, and then be later impeached for dishonesty.
Admit the claim, and then well, admit it.
IF some of the content is serious enough, it could surely be a foundation of a congressional hearing or perhaps even impeachment process? How can he respond to the questions there?
In any scenario... I can't see how he could "win" this? (Unless he is really innocent of anything questionable of course)
Which is why Trump probably won't be impeached, or even seriously censured - doing so would only cause an insurgency by Trump's supporters and it would weaken Republicans in the face of the Democrats. It would be suicidal.
Republicans (this was a party line vote) say it is unfair that Google and Facebook have your personal information and use it for ads but why can't ISPs have that and also sell it? One big major reason is people sign up to Google and Facebook for the purpose of sharing and agree to their ads in exchange for a service. Google built the most powerful search engine and Facebook built the social graph. Google/Facebook built value and they only use your info to target ads to you, they don't sell it because others would do the same. They sell ads and people use them because they have info on you, not necessarily to sell off to others.
If you ask me it is unfair for republicans to legally allow ISPs to do the same because we expect privacy from ISPs in ways we do not from Google and Facebook. You can choose not to use Google or Facebook but you cannot choose your ISP/broadband provider. In my opinion this is like letting someone view your mail, read it and then sell information about you.
It is also an unfair competitive advantage for ISPs above all because they can place ads on any website if they want or track you across all sites not just like Google/Facebook which are huge but only see a portion of what you do. ISPs built no value product like a search engine or social graph for this purpose, they should do that if they want access like Google and Facebook. It seems almost like the GOP are harming innovative companies and rewarding/catching up non-innovators. I bet broadband companies/ISPs won't even use the profits to improve broadband and rollout gigabit service for real. It is a rewarding of lazy semi-monopolies over innovative companies and products.
Republicans also control the FTC not the FCC so they want all control to fall to the FTC instead. It is both a power grab and a bending over of all their constituents.
Most of all, it is also another step in dismantling net neutrality as FCC protected that by categorizing the broadband/ISPs as a common carriers and they want to sap the FCCs power in that regard.
Some countries have decided to face fines and sanctions to keep data retention in their national law as long as they can but most have promptly killed their local implementation of data retention.
[1]: http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:320... and https://en.wikipedia.org/wiki/Data_Retention_Directive
Source?
2. The very notion of a lookalike audience (and its unfortunate popularity amongst the growth hacker types) is an immediate giveaway that FB/Goog are never going to take this tracking down even a single notch.
3. Google/Facebook actively encourage you to be always signed in to their networks
4. Google Analytics has more or less ensured that they see a BIG portion of what you do. Besides, just imagine a casual web user jumping from site to site. How many times are they going to land on a site which doesn't use Google Analytics?
5. Android is the dominant phone OS
6. Let us not even get into the infuriatingly patronizing attempt by Facebook to become an ISP in countries like India. Why didn't FB try to pull that stunt in a more developed country? Food for thought!
This is a case where pretty much every company is in the wrong.
>> People are missing the point on the bill, it isn't just horrible for privacy.
However, the overall point you make is true.
even before you join FB or even if you never join.
Wait, what? Never visited the US but I assumed you had the choice of providers, just as most of the rest of the world where internet is available.
I see the reason for this to allow ISPs to have infrastructure to collect data setup, so when some organization needs* access to it, it's already there and easily accessible.
* for national security or whatever
This is because the national telco monopoly was broken up early into horizontal companies, rather than the "local loop unbundling" system in the UK (and I think other European countries).
LLU is a little strange because the lines are still owned by one company, so there's no choice of who does the maintenance and installation work (it's always BT OpenReach in the UK). But you do get a choice of what happens to your packets after they reach the DSLAM.
Unless you live in a major metro area - where you might, if you're lucky, have two choices of cable provider - there's a reasonable chance you only have a choice between cable, ADSL over some neglected wiring to an exchange a few miles away, and satellite. So if you depend on having fast internet speeds, you're stuck with the one cable provider.
Also - it's not particularly uncommon that an apartment block is wired up for a specific provider and unable to get service from any other provider due to a monopoly contract the building owner signed. Then you also have that happening over entire areas in places, just with local Government signing the contract.
The best course of action on the part of the U.S. likely would've been to permit the AT&T monopoly to continue. AT&T's regulatory treatment at the time permitted it to set prices to ensure a reasonable return on its capital investment. That had the effect of creating an enormous incentive on AT&T's part to invest a lot in its network (the more capital investment, the more profits). As a result, the U.S. telephone system was extremely high quality for its time.
Keeping AT&T would've given people what folks on HN want: a wired network that was expensive built gold-plated.
This is an effective way to maximize profit at little to no cost and is not uncommon in many markets around the world.
Yay, free market.
Even in rural Cedar City down south you have about 5 ISP choices. Though the highest speed looks to be TDS at 300MBps
Ive usually seen 120 MBps/$80 as the max but in my aforementioned building Comcast sent us a friendly email saying we were being upgraded to 200 MBps at the same rate, the day before our building announced the second ISP available at 100 MBps/$69. That smells to me a lot like they always had the capacity, but never turned it on until they finally had competition.
Note: It looks like Comcast has started offering a 2000 MBps connection for 299 a month. Maybe it was just a coincidence that they were adding new capacity at the same time
And as for creating value, ISPs invest billions of dollars a year building physical infrastructure. It sure as hell wasn't Google that built fiber to my house.
Overall broadband ISPs have failed at gigabit rollout for which they got tons of federal funds to do so. They don't put in enough in profits back into R&D or rollouts unless to an extremely wealthy area.
If you truly think broadband/ISPs care whether you have good internet or fiber then my guess is you live in one of the areas that had some competition. Otherwise they sit back on their semi-monopolies and stop innovation in broadband where they can to milk it. They could drop channels on digital tv and use more for internet and slowly have started to do that, but they are drawing it out big time. Their "big innovation" in the last decade was data caps and constantly messing with VPNs and other valid uses of broadband. They only innovate when they have a chance to further their monopoly or stranglehold on an area.
I'd say broadband/ISPs, once a bright shining innovative force in the 90s/early 2000s, have gotten lazy and are actively slowing down innovation now [1], and this lobbying to get a competitive advantage over Google/Facebook is a good example of non-innovation but using the system to slow innovators and innovation in broadband/internet services and products.
[1] https://arstechnica.com/information-technology/2013/12/why-c...
A 10x improvement is pretty great. I can't say Google is 10x more useful than it was ten years ago. And many other products by "innovative" companies (e.g. Facebook, MacOS, Windows), have gotten worse in that timeframe.
> Overall broadband ISPs have failed at gigabit rollout for which they got tons of federal funds to do so.
Factually untrue.
(The Ars article elides an important factor: upgrading speeds isn't just a matter of flipping a switch. You need to upgrade expensive head-end equipment, the lines from the headend to the core network, split nodes, etc. All of this is very expensive.)
>> Overall broadband ISPs have failed at gigabit rollout for which they got tons of federal funds to do so. >Factually untrue.
Factually untrue that it is factually untrue.
To this day most people don't have access to fiber[4], that is a failure. Maybe you do but most do not.
In the late 90s there were $400 billion given to broadband providers and the job was by 2006 for everyone to have fiber running to their homes. They stole that cash [1].
Verizon is horrible at keeping it's promises on fiber [2][3][4]. Verizon has sucked from the time it was BellAtlantic, for a doctor/pharma web network right when the internet started going we used to run fused ISDN lines for 128-bit (two 64bit lines together, one for video one for data) and they didn't get one line right the first time and rarely did it meet bandwidth needs. Whenever there was a Bell Atlantic truck around everyone's network went down. They are still as sloppy and slacking big time in gigabit/fiber. But then again so is every broadband mini-monopoly.
I live in Chandler, AZ one of the first places to get cable internet with Dimension/Cox. They were innovators then, now they drag the rollout of gigabit and Cox's gigablast stopped when Google Fiber slowed down. They can't even get fiber to the place that was one of the first cable internet locations (primarily because of Intel being here).
Telcos have purposely failed on the fiber/gigabit rollout. I guess we will have to agree to disagree. Broadband/cable companies were innovative in the 80s and 90s and maybe early 00s, now they are just bean counting and sitting back with their mini-monopolies comfortably slowing innovation in the US.
[1] http://newnetworks.com/ShortSCANDALSummary.htm
[2] https://www.techdirt.com/articles/20131012/02124724852/decad...
[3] https://arstechnica.com/tech-policy/2017/03/nyc-sues-verizon...
[4] http://gizmodo.com/after-billions-in-subsidies-the-final-ver...
That number is fictitious. I've addressed it at length here: https://news.ycombinator.com/item?id=7709556. The number is built out of two pieces:
1) Comparing the company's rate of return to that of utilities like power companies, and calling everything above that "excess profits." (By that analysis, taxpayers are paying Google and Facebook enormous subsidies.)
2) Calling accelerated depreciation of telecom network infrastructure a "tax break," while ignoring that accelerated depreciation made total sense in the 1990s as telcos upgraded their core networks with fiber.
> Verizon is horrible at keeping it's promises on fiber [2][3][4].
Everyone is horrible at keeping promises on fiber. See Google halting Fiber rollout. Beyond that, the question is: what's the harm? Did New York pay Verizon a bunch of money for building out broadband in New York City, only to have Verizon renege on its end of the bargain?
Here is the lawsuit New York City filed against Verizon relating to the deployment: https://consumermediallc.files.wordpress.com/2017/03/verizon.... Here is the underlying franchise agreement: https://www1.nyc.gov/assets/doitt/downloads/pdf/verizon_nyc_.... Read both, and point me to the big check New York wrote to Verizon to get FiOS service. Or even the big tax break. You won't find either. If those things existed, they'd be in there (because "and then they pocketed the money we gave them" makes for a way better lawsuit).
What you do find is that Verizon got to pay New York millions of dollars + 5% of gross revenues for the privilege of spending another billion dollars building the FiOS network out to New York City.
So on the moral outrage meter that ranks up there with Lenovo teasing the Thinkpad Classic but never releasing it. People aren't outraged because they pre-ordered and Lenovo kept their money. They're outraged because they want it and they can't get it.
> They are still as sloppy and slacking big time in gigabit/fiber.
They've deployed fiber to 15x as many subscribers as Google, and are rolling out near-gigabit (provisioned at 1024 mbps but marketed as 750 mbps) throughout the FiOS service area through 2017.
Again I think we'll have to agree to disagree.
We get it, you love Verizon, I hope you are invested with this much love for a broadband company that will take you at every turn they can. But have fun if you ever need a line installed from them or make a deal with them to do so across a city.
Do you have fiber from Verizon? If not maybe take a step back.
Fiber has been largely danced around by broadband/ISPs and even Verizon is far, far from coverage that people expected by 2017.[1]
In most cases they were offered monopolies over areas in agreement with them wiring it and offering good service to all, an immense benefit for monthly customer subscriptions. They have failed so far, it is 2017 and barely anyone in the US has fiber or gigabit (<10%). We are letting the broadband/ISPs drag us down, we should be leading the world in broadband/gigabit/fiber.
Verizon doesn't actually want to run fiber.
The simple fact is that Verizon has been trying its damnest to get out of the wired business altogether. Back when Ivan Seidenberg was in charge, he made a giant bet on fiber, which is why Verizon became such a national leader in broadband with FiOS -- a service that people really seem to love. However, Wall Street has always hated it, because it's capital intensive, and Wall St. recognizes that without any real competition in the broadband space, Verizon can avoid investing in such infrastructure upgrades, and just swim in larger profits while America's broadband infrastructure suffers and falls further and further behind other countries. Once Seidenberg left, the beancounters quickly took over and looked for ways to stop all that investment. Why invest in the future if there are no competitors to push you to do so? [2]
Verizon has tricks.
A decade ago, we wrote about how Verizon had made an agreement in Pennsylvania in 1994 that it would wire up the state with fiber optic cables to every home in exchange for tax breaks equalling $2.1 billion. In exchange for such a massive tax break, Verizon promised that all homes and businesses would have access to 45Mbps symmetrical fiber by 2015. By 2004, the deal was that 50% of all homes were supposed to have that. In reality, 0% did, and some people started asking for their money back. That never happened, and it appeared that Verizon learned a valuable lesson: it can flat out lie to governments, promise 100% fiber coverage in exchange for subsidies, then not deliver, and no one will do a damn thing about it. [2]
I am with the people that want fiber across the US, beancounters and investors of Verizon want the opposite. I guess you are in the latter category?
[1] http://fiberforall.org/fios-map/
[2] https://www.techdirt.com/articles/20131012/02124724852/decad...
> Do you have fiber from Verizon? If not maybe take a step back.
My parents have had FiOS for more than a decade, and I've had it for several years in four different houses and apartments. I always get 5-10% over the advertised speed (even during peak times) and my ping times are similar to the Cogent business internet we have at work.
> They have failed so far, it is 2017 and barely anyone in the US has fiber or gigabit.
About 11% of all U.S. broadband connections are fiber: http://www.oecd.org/sti/broadband/oecdbroadbandportal.htm. 25% of the country can get it: http://broadbandnow.com/Fiber (FiOS's uptake rate is less than 40%). We're doing better than most of the other big OECD countries. Canada is at 8%, France at 6.5%, Germany, Italy, and Austria are under 3%. Spain, South Korea, and Japan are the only big OECD countries doing better than us.
Moreover, the overall statistic ignores the fact that broadband is largely a state level issue, and deployment varies a lot by state. E.g. Maryland, where I live, has about the same population and population density as Switzerland, and has similar fiber availability (over 60%).
> A decade ago, we wrote about how Verizon had made an agreement in Pennsylvania in 1994 that it would wire up the state with fiber optic cables to every home in exchange for tax breaks equalling $2.1 billion.
There was never any "tax breaks": https://news.ycombinator.com/item?id=8401102. (Read the thread--the guy who came up with the $400 billion and $2.1 billion numbers explains how the number is based on calculating "excess profits" compared to regulated rates).
Consider yourself lucky then. You might view it differently if you are in an area that is yet to have fiber. Here in AZ, Tempe and Scottsdale have it but nowhere else. Cox put gigablast/fiber in only in response to Google Fiber and has subsequently stopped after Google slowed, so maybe we have different experiences. It is very frustrating. You are in a very small state that is near D.C. which is one of the heavy focuses of fiber (New York, DC, CA, WA, etc). If you move to an area where it isn't as prevalent, you might not be so pro-broadband/ISP providers.
> You can't "agree to disagree" about facts. You're throwing out assertions that are simply false (e.g. that companies agreed to deploy fiber in return for being granted monopolies).
I should have said semi-monopolies like I did initially. In many cases this is true simply by being the one to run the lines and discouraging competition. In many, many cases we only have one provider for most of the US. Providers are also heavily against municipal broadband and other competitive services, only Google Fiber really scared them into increasing rollouts.
I can agree to disagree, many of the things I have posted back up my main point that fiber rollout is slow, there have been lies and incomplete rollouts and it is holding us back in the US. According to you broadband/ISPs are doing exactly what they can and pushing the limits of capabilities and innovating like they did in the 90s. That may be the case in Maryland in your area, I have a differing opinion and frankly I have never met someone so pro-broadband/ISP/Verizon unless they were investors or worked there. I do hope broadband/ISP providers return to their innovative glory of the 90s and early 00s. They are failing in that regard today.
This article is actually somewhat encouraging, in that it displays the limits of the laws as they exist today and will apparently remain. You cannot go buy the browsing histories of specific individuals - enemies, employees, etc. Many of the ridiculous "sky is falling" leaps of fact and logic that have been portrayed in the media to get views (ironically so that they can make money from displaying targeted advertising on articles decrying it) have been disproved by this article.
Until mid 2016 ISP privacy abuses could be dealt with through the FTC, basically the same as privacy abuses at most other internet businesses are dealt with.
In mid 2016 a court ruled that the FTC did not have authority to deal with these issues for common carriers, which ISPs had been since being reclassified as part of the 2015 net neutrality rules. The result was a significant loosening of privacy regulation for ISPs.
The FCC rules would have undid that loosening.
You could buy pretty detailed documentation, including internet usage history, from IRC bots on the shadier shades of the doxxing market 15 years ago.
Not everybody's would be readily available or as detailed but if you could afford it you go for on-demand doxxing, the more detailed and the more high profile target the more expensive. Among the data extraction methods used, one was gaining remote access to target's computer, collect the data collected, curate it and sell it.
So yes it was possible to buy browsing history of specific individuals 15 years ago, so I would be surprised if it wasn't still possible today. Don't ask me where to look, I haven't gravitated under these shades since 15 years ago.
One cool thing I heard Apple talk about[2] was injecting noise or subsampling to help mask individuals. Although, I don't see that mentioned in their public page on privacy[3] or their whitepaper[4].
[1] https://youtu.be/Erp8IAUouus
[2] https://www.wired.com/2016/06/apples-differential-privacy-co...
[3] http://www.apple.com/privacy/approach-to-privacy/
[4] http://images.apple.com/business/docs/iOS_Security_Guide.pdf
When doing this procedurally, it's amazing how few data points are needed to pinpoint a specific person.
Oh, and of course they found and deanonymized some German politicians and their staff.
So while it might not be possible to deanonymize and "expose" all of Congress, if you really tried and got some money to actually buy the data, you will be able to at least deanonymize some of your targets, especially if the data originates from a man-in-the-middle like an ISP and not just some random ad network/tracker.
https://dts.podtrac.com/redirect.mp3/traffic.megaphone.fm/BU...
But I can buy anonymized browsing and demographic data for the downtown DC zip codes, and de-anonymization is not particularly hard when you have so much information to work with. Zip code, gender, and birth date together are enough to get you to 87%.
They probably don't understand or know that, though, so why not give them a scare?
Well, NSA staff did LOVEINT. Maybe some ISP staff will want to profit.
Edit: But I wonder how it'd be possible without self-pwning.
While the publicity pieces may be poorly stated, (in particular one should never use the word "individual" in any context) the desired teachable moment is based on buying data or access to a segment of at least 500 individuals based on factors like travel habits, income and demographics. (In some proposals that segment could be as high as 5-10K as I see mention of congressional aids and the like.)
That desire could be quite easy to achieve and quite hard to block without upsetting the intended market for ISP data since being able to buy affects on segments down to such granularity is exactly how the market works.
So wouldn't it be illegal for sites to sell visitor information (email lists)?
Then again, I'm pretty sure google does not know about what people search on bing or yahoo, and it seems to me that what you search on google appears in the URL so your ISP would know.
Well no, Google searches go over HTTPS, so everything but the host is encrypted.
Hasn't this been shown repeatedly that it is not only feasible but also quite easy ? It seems to me that the verge missed the point here.
http://whois.arin.net/rest/net/NET-143-228-0-0-1/pft
http://whois.arin.net/rest/net/NET-143-231-0-0-1/pft
https://en.wikipedia.org/wiki/Wikipedia:Congressional_staffe...