Hacker News Reader

Comment by pandabear187 | Hacker News Reader

I have insider knowledge as I used to work for one of these companies.

Depending on the product you purchase the data comes from multiple sources. Also these companies have sophisticated machine learning capabilities to build a profile based on various attributes found in seemingly unrelated pieces of data.

So the list consists of credit reporting agencies, public records, your online profiles with public access, court records, aggregators like LexisNexis and dozens like them.

This heavy lifting you speak of is done differently by each company and consists literally based on multiple sources to enrich your profile. These companies spend millions on data and engineering and make even more, and whatever preconceived notion you have about courts ordering to seal your records, it doesn't happen in a centralized fashion, you would need to contact each data vendor individually to be removed. But it would be like playing whackamole.

View on HN

Thank you for the reply. Can you actually give some kind of a list though? The entire problem here is everyone explains the how but no one is willing to explain the who. I certainly understand it's "multiple sources", I'm asking who are these sources I keep hearing about. There can't be nearly as many sources as there are sites who buy from them. If you'd like to not name the company you worked for yourself then could you at least please list as many other ones as possible? That would be far, far more helpful than just saying they use machine learning and that they use multiple sources, etc.

There is no such public list. Most of these companies are privately held, their methods are trade secrets, and there is no real form of legal recourse. Your best bet would be to buy a book like this: https://www.amazon.com/Hiding-Internet-Eliminating-Personal-... and follow the recommendations, but even that is only likely to be half-effective.

Much of the public information is mined from sources like credit headers, your court records, utility bills, property and tax assessment records, voter registration lists, motor vehicle registrations, etc.

Unfortunately, the legal and technological landscape is such that 'hiding' from these kinds of services is effectively impossible.

> There is no such public list. Most of these companies are privately held, their methods are trade secrets, and there is no real form of legal recourse.

How can there not be a public list of these data miners? When e.g. a court needs to control someone's information surely they know who these people are and they can let them know? Is there a secret list in every courthouse or something?

Or when someone wants to start another one of the higher-level companies -- how do they know which core aggregators to buy from? If that's a secret then how would they find out? Surely someone's gotta be willing to tell?

> When e.g. a court needs to control someone's information surely they know who these people are and they can let them know? Is there a secret list in every courthouse or something?

The courts don't control information in that way.

That's surprising. So what happens when there is a high profile case and the court fears for the jurors' lives? Or the case itself involves someone whose life will be in danger afterwards? If anybody can find out where these people live then they're toast. The courts have to suppress the information legally somehow, right? If not the old information, at least they need some protection against mining of new information after the subjects move or change their names, no?

Are you literally saying they have no way to order all the first-level companies to stop sharing data on someone?

> So what happens when there is a high profile case and the court fears for the jurors' lives?

Jurors' identities are not generally a secret.[0] There are exceptions, but those exceptions do not extend to wiping that person's data from things like pharmacy and gas station reward card databases.

Honestly your entire premise shows a lack of understanding of how the criminal justice system works.

> Are you literally saying they have no way to order all the first-level companies to stop sharing data on someone?

Sue all of them individually, win each case, and have them ordered to stop collecting data on you. Something tell me this is tantamount to "don't use a computer or a credit card. Ever."

[0] http://www.legalmatch.com/law-library/article/public-access-...

> Jurors' identities are not generally a secret.[0] There are exceptions

I was talking about the exceptions. If there are exceptions, a way to handle them must exist, is all I was saying.

> but those exceptions do not extend to wiping that person's data from things like pharmacy and gas station reward card databases.

I was asking about the companies who obtain this original information, not pharmacies' or gas stations' databases themselves. I feel like you're not understanding my question?

> Honestly your entire premise shows a lack of understanding of how the criminal justice system works.

Quiet likely (I'm not claiming otherwise; I'm not a lawyer and I haven't exactly been involved in legal proceedings) and I didn't claim otherwise. Also hardly undermines my point. Like I've said in some 3-4 other comments, someone who e.g. starts a new company like InstantCheckmate has to know whom to buy the data from -- like I said, there's no way all of these companies contact all grocery stores and all doctors. That's insane. Someone's gotta be doing the heavy lifting and making money off it. I'm asking who this is. I asked this in the original post. If the court example is wrong or otherwise bothers you just ignore it. If someone not already involved in the business knows to contact these companies to obtain information, someone must know who they are, is all I'm saying. Otherwise they would not exist.

> I was talking about the exceptions. If there are exceptions, a way to handle them must exist, is all I was saying.

There is. They put them in a hotel, under police guard, for the duration of the trial. https://en.wikipedia.org/wiki/Jury_sequestration

What if I told you there are countries where you can use a credit card, and still have reasonable expectations about privacy, where the information gatherer must erase personal information on demand and bears the burden of erasing it and notifying everyone they sold the information to erase it too?

id like to know which ones? id consider relocating.

1. They don't know, but assume some Western European country fits that description.

2. They misunderstand German privacy laws.

> Are you literally saying they have no way to order all the first-level companies to stop sharing data on someone?

This is exactly what he's saying. There is no central node of control for this kind of information. You are operating from entirely unjustified assumptions.

For very high profile cases, a jury would be sequestered, meaning they are housed somewhere and not allowed to interact with the public. If they were in danger, their would be guards protecting them.

After the trial is over, though, they are on their own. They will not continue to be protected, and could certainly suffer retaliation. It sucks, yes.

http://criminal.lawyers.com/criminal-law-basics/sequester-is...

a court needs to control someone's information surely they know who these people are and they can let them know?

I think you have a misunderstanding of what a US court can do. A court can only tell a specific party to take some action, and generally only if that party is somehow related to the legal action (such as being a defendant). Generally, there is no judgement that a court can make that can effect unnamed parties (unless they are John Does, which later have to be named).

Theoretically, you could sue each company with your data, and a court could tell each of those companies to remove your information. But it would have to be for each one, and the judgement is only binding on those companies.

Surely someone's gotta be willing to tell?

The techniques used are generally trade secrets, and amount to competitive advantage. There is little incentive for a company to reveal this information (or for an employee to do so, and thus open themselves up to legal liability).

What about this though:

>> Or when someone wants to start another one of the higher-level companies -- how do they know which core aggregators to buy from? If that's a secret then how would they find out?

   > If that's a secret then how would they find out?

You don't. I've worked in data acquisition in the past, both buying data and selling it. Sometimes as the original source of truth and sometimes as a middleman that does data cleaning, standardization, appending (from other sources), then selling the derived product downstream.

Companies in that space guard their upstream sources quite heavily, because they don't want to be cut out of the process. You won't find a centralized list of independent data feeds and providers specifically because of that. In one scenario, we were dealing with a substantial rate increase from one supplier. We spent time attempting to source an alternate supplier of that particular type of data, and could only find sources that were several months more stale than we were currently getting (i.e. these people were getting the feed several hops after we were). In the end we paid the rate increase because we couldn't find an alternate source that was as close to the original data provider as our current source. And without knowing who the original data provider was, we couldn't go around our supplier.

The lack of a centralized directory isn't just done to make things opaque for end users, it's done to make things opaque for business competitors as well. It's an industry that's very, very reliant on networking and introductions.

Edited to add: You're also asking a lot of people in here to name specific companies even if they can't give you huge lists. This space is super heavy on NDAs (and trigger happy on enforcing them). If you've actually worked in it, there's simply no way you're able to name drop legally.

+1 thanks for the explanation!

And regarding this:

> Edited to add: You're also asking a lot of people in here to name specific companies even if they can't give you huge lists. This space is super heavy on NDAs (and trigger happy on enforcing them). If you've actually worked in it, there's simply no way you're able to name drop legally.

I understand that an NDA would prevent you from naming your own company or your suppliers and clients, but surely it doesn't prevent you from listing some other companies in this space that you know of (including but not limited to your competitors)? I don't understand why you shouldn't be able to name any company just because you've worked at one of them.

Every company I know in the space is a company that we've had at least preliminary conversations with (seeing if there's any potential relationship to either purchase from or sell to that company).

Just having that conversation required getting a mutual NDA in place, since the conversation involves revealing your capabilities (even if not your sources). And that's assuming you're even aware of all the NDAs your company has signed with other companies, which isn't always the case. Speculation or name dropping in public could violate an NDA you're not even aware of, then you find yourself having to defend your speculation as just that, rather than as revealing proprietary knowledge (that you didn't actually have but your company did).

At the end of the day, it's easier to default to speaking in generalizations rather than risk the potential repercussions of not doing that. :-/

Okay, I see. But somehow someone in your company found out about them before they could sign mutual NDAs, right? How does that happen? Do you just need a higher-up who's friends with the right people?

i imagine its similar to how drugs and weapons dealers meet. when you know seedy people, you get connected. birds of a feather and all that.

Imagine the legal liability you'd incur if the layperson could track the source of inaccurate data, then proved in court it kept them from getting a loan.

Those are provable damages, and maybe slander if they can convince the court they're a publication

Another reason to be NDAed up.

> Just having that conversation required getting a mutual NDA in place, since the conversation involves revealing your capabilities (even if not your sources). And that's assuming you're even aware of all the NDAs your company has signed with other companies, which isn't always the case.

Interesting. Eight years ago I worked as a buyer for the aggregator with the largest criminal database and I don't remember having to sign an NDA during those sorts of talks. It's possible I've forgotten, but I think it was more a matter of people wanting to know our coverage as much as we wanted to know theirs.

As an aside, the value of those talks wasn't usually in acquiring the data, except maybe in the short term. We always preferred to go directly to the source. The value was simply in learning that the data from a specific source was even available. In a few instances, that got pretty frustrating, since I knew that the seller of the data was scraping a given court's website against that court's wishes (and the TOS on the website), with all associated problems with accuracy and ethics that entails.

Go to an industry site like www.napbs.com and look around.

> Theoretically, you could sue each company with your data, and a court could tell each of those companies to remove your information. But it would have to be for each one, and the judgement is only binding on those companies.

As an individual, that sounds like a lot of work for little gain, but if there were pre-filled out forms, and all I had to do was put in my name, I'd be willing to file lawsuits to get my name removed.

What about class action? Could you make a class action against those who have sold your information?

A class action is an action where there are a large, possibly unknown, number of plaintiffs suing a single group of named defendants. With the thought process being that if a single case against a named defendant is won, then that same outcome would happen for every other plaintiff, and thus doing it once saves the court time.

What you're suggesting is more like a John Doe case where you sue a number of unknown entities, and that can be won, but at some point, the plaintiff has to name the John Does, so that they can defend themselves.

Couldn't you start a class action against one named defendant, and use the discovery process to uncover all the other unknown-at-the-time-of-filing defendants?

I'm pretty sure that if the named defendant coughs up an NDA that prevents them from disclosing the names of their business associates to a court, the judge is not just going to say, "I'll allow it."

It has successfully been done in the past, so yes, you could do that.

>Theoretically, you could sue each company with your data, and a court could tell each of those companies to remove your information. But it would have to be for each one, and the judgement is only binding on those companies.

I'd expect going after such companies to be the state prosecutor's job.

If there are many companies, surely it would be in their interests to leak one another for competitive advantage?

You grossly underestimate the scope and power of the United States judicial system, which does not ever have a need or desire to "control someone's information."

> When e.g. a court needs to control someone's information surely they know who these people are and they can let them know?

The companies which furnish personal data aggregated from courts are legally required to stay on top of out-of-date records and purge records which are inaccurate. (it would be completely untenable for things to work the other way, with every court and agency which made records available being required to reach out to every recipient of the data. For one thing, in some cases the data can legally be resold.) They can be held civilly liable for distributing false information and might also be in breach of their agreements with the agencies which give them access to the data.

The urgency with which this is required under the law depends on the use to which the data is being put. It's really important to keep data used in pre-employment reports up to date. Marketing data can generally be full of garbage.

This is all generally governed in the United States under the FCRA. It's not an area of the law that you're going to get comfortably familiar with in just an afternoon of reading.

I have a friend who is super paranoid about privacy stuff. He was even able to get his utility bills to use a PO box rather than his home address, which he said was extremely difficult.

Depends where you are in the country. Many rural places don't have home delivery and only have P.O. Boxes. Home delivery from Amazon, etc. requires a bit of work.

I can give you one very specific answer in that my late father decades ago worked in IT for a collections agency and one way that biz works is they pay a fixed amount to buy some companies 120+ day accounts receivable file (the feds and every state highly regulates this and its highly variable and complicated, but this is the simplified version...) and this gave them a vast pile of records and legal ability to collect the debt. Now obviously they got most of their revenue by annoying the heck out of debtors right up to the legal limit. But they're leaving money on the table if they don't resell all the data they can legally sell. So sure, even 20+ years ago collections agencies were uploading records to various data aggregators in exchange for a check. Not just the original debt but followup activity and intel WRT the collections process. I would imagine that's only increased over the last couple decades. Most of these data transfers were two way streets, not simple data for money trade, and obviously they maximized profits by leaning the hardest on the people most likely to actually pay up, they did not waste time on debtors known to other collections agencies as hopeless deadbeats.

I can provide all the sources used for the company I used to work for, however that's irrelevant if you as you said wanted to learn how to fish.

Any and every company selling data and there are thousands is in fact a source you would need to deal with to be removed from the sites that sell it. If you think this answer isn't specific enough, you're not going to achieve anything by me spoonfeeding you info for one such company.

>I can provide all the sources used for the company I used to work for, however that's irrelevant if you as you said wanted to learn how to fish.

Personally, I think this should be a requirement of the government: "HERE IS A LIST OF ALL PEOPLE THAT ARE COLECTING AND SELLING YOUR PERSONAL DATA, CLICK THIS BUTTON TO DELETE YOUR RECORDS" sort of thing... it should be a mandated public service of regulation.

Then, they have a list of all opt-outs from which companies you may have selected to opt-out from, and you can simply send them any further contact from the opted-out companies you may receive and the company gets a fine, you get a compensation fee...

2 things, 1.)if you make government enforce this, then we are all going to be paying more and things taking longer. This will push the society toward a fascist or communist states where the government is the all powerful and might beast. I do not like this.

2.) in western countries there are already laws that allow you as the consumer to remove any personally identifiable information from being shared. The ownus is on YOU to go do that, it's not hard but will require to track and follow up every year for ever. However you also have a choice of companies that you provide the information to, and what you provide. Any credit agency when you apply for credit asks if you would allow them to resell this. Opt out!

The fact that in most cases people are to lazy to read and understand what they are provinding and for what purpose is the real issue here. Government is not going to make things better or more secure or even be able to enforce this type of governance. Only you and the lawyers can do this.

I don't think it's fair to call people lazy for not reading every line of an exhaustingly-long TOS. Also, it isn't fair to expect every consumer to fully understand the legalese that they are reading if they do decide to read it end to end. "Don't click 'I Agree' if you don't know what you're agreeing to!" is unrealistic.

This is sort of how European data protection is supposed to work, although you have to maintain your own list as there is no central one. And the enforcement is intermittent.

(Data subject requests are a very powerful tool)

It used to be like this in many European states back in the 80's. If you wanted to keep a computerized database of personal information, you had to apply to the state for a permit, so there was central control of all the lists. Of course this was extremely difficult to enforce... Eventually that type of legislation was replaced by the EU data directive.

e.g. In Sweden: http://www.datainspektionen.se/om-oss/historik/

If I was a company, I would be writing scraping engines that scraped

* police 2 citizen (a platform many counties and municipalities use to report crime and accidents to the public)

* any public facing dataworks plus web application (or whatever various other municipalities/counties are running): the one for the county I live in lists the arrestee's employer

* district level and state level court dockets

* real estate records, which also link up to tax bills

* public voter records

Not surprisingly, only the information I've ever listed on my voter ID has ever showed up in Intelius/LexisNexis databases.

> There can't be nearly as many sources as there are sites who buy from them.

Sure there can. The sources include state, county, and local governments. There are a lot of those.

If you ever buy anything with a CC, use a "membership" card anywhere, or have anything delivered, or for that matter purchased online, then there's a greater chance than not this data was shared by the merchant, transaction provider and/or the credit card company themselves.

If you have an online profile, with any friends that aren't paranoid, and allow your friends to see any private information, then this can be collected/correlated by the various bot farms.

As a person who used to be involved in that industry, do you try to protect your data more? Or is it just a losing battle?

Yes, I protect the shit out of my personal info. I have introduced a seeding mechanism to figure out who is sharing my data whenever I sign up for things. Public records are different and you have to create a Trust or a company to hide behind to not be personally listed.

I would suggest anyone wanting to not be exposed to never use your real name or address online if you're not confident they won't share it. Introducing typos or initials or use a nickname for online orders that allow to list billing address separate from shipping address. Billing data typically has a better chance of not being resold, but there are no guarantees, read their TOS if not sure.

I really think you should do a how-to / AMA

Financial ignorance is an exceedingly exploited issue in the US, and I do not think anyone is doing anything about this.

Would you be willing to write up a guide of some sort about the steps that someone could take to protect their information? Personally, I would be willing to pay for something like that.

Are any of these companies operating in Europe and/or with mostly European data?

Reply on news.ycombinator.com