How Palantir pushed into policing | Hacker News Reader

How Palantir pushed into policing

wired.com

78 pointsmcone15 comments

Nice article. Palantir's practices seem to follow Thiel's master plan for world domination to a T -- establish monopoly then protect it by any means necessary: obfuscation, proprietary APIs, overly complex user interfaces, and finally claims of sensitive secret company technology that must be kept secret or risk compromise, perhaps causing harm to the US military and lowering our defenses against terrorism.

What a crock. There's little doubt that 99% of Palantir's infrastructure is commodity open source, trivial to access, monitor, and replace. It's absurd that any customer would pay a premium for proprietary hardware and system support, much less be told that the system disallows even the shallowest oversight of the company's data and algorithms.

Unlike military contracts, police intel must be transparent to oversight and subpoena, or we risk further abrogation of our 4th, 5th, and 1st amendment rights. As an aide to the justice system, Palantir has a lot to explain, not just in court cases but on a continuing daily basis. Anything else violates the very fabric of America's principles.

>>establish monopoly then protect it by any means necessary

rent-seeker P Thiel

I don't really get this article. It doesn't really say anything new and continually throws out statements designed to get the dear reader on the opposite side. "Trump supporter Peter Thiel", complaining about how only Palantir can maintain their systems, tossing around how many millions of license plates it can store, complaining about how it doesn't operate well with other systems...this reads like a hit piece.

Yeah, Palantir's ecosystem is a closed ecosystem. But that's kind of becoming de rigeur these days for market leaders. Yeah, it sucks and ideally wouldn't exist, but don't paint it as some novel way for capitalists to fuck you over. That disingenuous approach does more harm than good imo.

I'm most definitely not a fan of Palantir ever since digging into them and bailing out of the hiring process ~five years ago bc it made me squeamish, but this article just doesn't really seem to say anything new or even show an even-handed introduction to the company.

>only Palantir can maintain their systems, tossing around how many millions of license plates it can store, complaining about how it doesn't operate well with other systems...this reads like a hit piece

Those actually sound like completely valid concerns when considering whether Palantir should be offered a concession.

Even better is to consider whether this concession should be offered in the first place in a supposedly free country.

It's worth considering that palantir, while big and spooky from a startup perspective, are a new kid on a block with some even spookier bruisers.

Palantirs competitors are folks like IBM, Lockheed Martin, and BAE.

We know a little about thier technology and what spooky stuff it can be used for because they had to publicly market it, because they have been trying to break into markets.

These other guys have comparable products. They have big data and network analysis tools. And more - IBM have some natural language analysis tech and curated data others can't touch for instance, and warehouses full of phds driving it.

But they also have better contacts, shady revolving door sales arrangements with budget holders, and moats of unintegratable software already installed. Would you bet your well paid post retirement consultancy job on the continued survival of Palantir or Lockheed?

I remember when the cooperation between hb garry and palantir on a tech demo came out in the aftermath of the anonymous hacks. They were scraping Facebook.

If Lockheed were doing a demo like that, it would have a million dollar budget paid by darpa and use some secret bulk intercept database from our nightmares, and the demo would be to a high level audience at a military base, not at a tech conference.

I don't like palantir, or the kind of things they are doing, but I like to remember they are a shrill little kid and the real scary stuff is behind the curtain. Imagine the most impressive deepmind demo done on a dataset of every email the nsa holds.

Peter Thiel's hypocrisy takes my breath away. He pushes to elect Trump and buys himself citizenship in New Zealand. He calls himself a libertarian and builds a company at the service of the surveillance state. He gets two degrees from Stanford and tells kids they shouldn't go to college. The guy's a fake, a phony, and he's doing harm to the country and ecosystem that allowed him to succeed.

The opening anecdote of this article is not a great look for Palantir. 100% consistency with the underlying database ACLs has long been one of the major selling points of the Gotham platform; perhaps the most important one.

The fact that (according to the FOIA emails, per the article) the gang detective was asking for a fix for the ACL on a specific record and not hearing back for months while the record remained accessible to other officers who should not have seen it is especially surprising, and bad.

Any "data" company must have full chain-of-custody for each one of their data points (where did we get it, how did we clean it, how would we update it, what was it previously, what has changed since X time, etc).

Otherwise they're just making accusations without any defense from repudiation.

Some of the wording in the article, like "data trove", implies that Palantir has a central repo for all their data. From what I know, this isn't the case; a lot of government contracts mandate on-prem deployments, often on air gapped networks. The Police data might be an exception, though. I'm curious if anyone knows if this is the case?

The rest of the article paints the process in a light that has been mentioned many times: They are a very successful consulting business.

conscious that this probably sounds glib...

mandates and has are (in my experience) very different things in military systems. I would suspect that the police systems, as are a lot of military systems, far less likely to be airgapped mostly for purposes of convienience

I think the article is making the point that they are 'successful' in an economic sense but posing the question of whether this is an appropriate place for that type of success or the pursuit of that type of success. The potential for malfeasance seems particularly high.

that might be true, but my experience is that generally, these kind of software deployments are really actually on-prem and aren't using AWS (or similar) under the covers. This was the certainly the case for my last employer, that sold very similar software.

My remark at the end, I should have added emphasis on "consulting". my point was not so much that they are successful, but rather, they seem to operate as a consulting company, whilst portraying themselves as a seller of software.

fair and agreed. I have to be careful with Palantir that my personal disgust with them doesn't leak into my logic.

Are these systems really that sought after? Recently my team and I finished a responsive hazmat application which uses mapbox, elasticsearch, and some .Net for real time DB replication (about 5 seconds). Documents and data are ingested onsite and displayed on a map for public records request with secure bulk file download. Records date back to the mid-nineties up until now. So with some ML I've created a Palantir?

you did, but think about the cost of having competent senior software teams + it + product manager + project manager + ML engineers + data scientist and for a gigantic company buying palantir can make sense. for a software centric company it makes much less sense.

The devil is always in the details:

- You'd have to go through through a ton of red tape to sell your software to various govt agencies, in terms of getting approved as a vendor.

- Deployment complexity for various intelligence organizations is quite complex: air gapped networks, firewalls, and extremely locked down machines where getting a port opened is a bureaucratic nightmare.

- Data ingestion is a huge piece: don't forget that Palantir promises support for tons of different data types. Word docs, email, chat, GIS data, etc etc. hammering all this stuff into a relational schema and resolving entities (same person, different email) and de-duping is a nightmare.

- Have to support the inevitable waves of features that come in if basic functionality is successful.

The basic technical pieces, from everything I've ever seen with regards to palantir, are fairly simple. used to be Oracle, some Java services, some good frontend work with very good visuals. The real reason they've been so successful is less about innovative technology and more about managing operational complexity very very well.

Note that all this is secondhand information, I've never worked for Palantir and only know their product because we shared a lot of clients. I'd love to hear from someone at Palantir where I'm wrong.

Reply on news.ycombinator.com