Ask HN: Palantir Software Engineering
I'm trying to collect more info from anyone working (or has worked) at Palantir. Most of the reviews on glassdoor seem copy-and-pasted. If anyone has anything to say, I'd really appreciate it!
Like to what extent their software is used in planning drone strikes and other military operations. And what, exactly, Palantir knows about the number of civilians killed and maimed in these operations. Let alone how any of the engineers working there, behind their hoodies and their high-end headphones -- perhaps reading this very thread right now, as we speak -- personally feel about this situation.
https://en.wikipedia.org/wiki/Civilian_casualties_from_US_dr...
From my reading on Palantir they seem like a big data mining company. Sure, they can try to grab a ton of data and try to show some correlations but somewhere there must be someone reading that information and making decisions, right? It's not like Palantir is carrying out drone strikes by itself (or at least I can't find any reference to this).
I don't/never have worked for Palantir (I've heard it's very cultish fwiw so I've never applied). I have used their software though (not for drones), and it is actually pretty awesome, so I can't speak to their software quality but their product team was doing solid at least.
Also I really can't conceive as to why Palantir would have information regarding civilian casualties or anything else from their clients' data. There's about 0% chance that the government would allow a private contractor to hold that data (obviously if it were being tracked it would be on the copy running in production but Palantir wouldn't own that data).
Yes, perhaps it's output is used at some point when deciding something about a drone strike (although I can't find a source for this), but I'm not sure why the sentiment about this is so incredibly negative?
I think it has to do with a bit of iceberg effect. The part that shows up in news articles is all outrage inspiring so any mention of it becomes so just by association.
I think it's kind of similar how for some reason every time I say I like Hemingway the only response is "he was sexist". Sure, especially so if you reapply modern norms out of temporal context, however a) it doesn't add a lot to the conversation that wasn't covered in middle school english, and b) it's hardly a whole treatment of the originator of a rather large body of creative work. People jump to it because it's an easy opinion to hold and kinda makes it sound like they've done the thinking about it (which to be fair, maybe they have and have determined that to be the only relevant facet of the author).
Similarly the "Drones!" response is a) nothing that isn't true of many other systems, and b) not a full representation of Palantir's uses. It's hard to get voted down for being cynical on the internet, and it shows that the author has read a WashPo article or two about drones and Palantir, but again, maybe they've researched the company and the product fully and determined the only relevant thing is that there probably exist drone targeting packages informed by the use of the software.
What's the general gist of what Palantir's software does? What's the input and what's the output?
1. feed it a ton of documents
2. either procedurally (ideally) and/or manually (a la wikipedia but with a fast interface) link between them.
3. Also create entities, so like, maybe Bernie Madoff is an entity, and there are companies that are owned by him, those companies might have phone numbers. (totally bullshitting this example so please ignore legal, investigative, factual nonsensicality)
4. You keep putting all the docs you run across into the system as you go and one day you put in a doc from an investigation of a shady shell corp. One of the phone numbers listed is recorded as a Madoff company's number and you see that as you load in the doc. Either it's relevant for you or the next person to go look for info about Madoff would see it. That connection might never get made without the built up knowledge base and auto-associative kind of features.
I always looked at it like a super easy to use wiki or document database that had investigatory tools built on top of it. The search was really well done and following relationships was easy. Granted I was comparing the experience to other gov't software, the fact that it worked almost all the time and didn't drive me to script my interactions with it pretty much alone made it my favorite. So Palantir might objectively suck and I was just so mired in terrible that it looked relatively stellar, but I remember it as useful and easy to use.
As a customer of Palantir, am I responsible for creating the document parsers that understand the relationships between entities or is their software able to recognize data types in the inputs automatically? Is the software generic or customized for each customer that comes in the door?
[0] I know it's an odd description but I think it puts the right idea in your head. Look at the little desperado/mystery man icon: https://www.palantir.com/wp-assets/wp-content/static/techblo...
I just wanted to point out that their product is pretty awesome and I sometimes feel compelled to try to temper what I perceive as innuendo (vice fact) powered comments towards the national security community. There's plenty of real cause for anger (including billions wasted on software that doesn't work), a really good network analysis tool isn't one. Perhaps I saw innuendo where none existed, but certainly agree with what you've said here.
[1] http://www.washingtontimes.com/news/2012/jul/16/military-has...
Or in identifying "terrorist" targets generally?
1) Drone Program / Terrorist Hunting / intelligence = bad therefore companies helping with intelligence gathering / processing = bad/nefarious?
2) Software used by the government to execute programs some might disagree with is tainted/bad therefore we should boycott it/protest against it?
There are hundreds if not thousands of government contractors providing similar services to the Govt, albeit quietly and behind the scenes. The only reason Palantir gets a lot of press is they do it better than most other people and they aren't based in the DC area. If we're supposed to "do something about Palantir" what are we supposed to do about the rest of them?
Palantir et al. aren't in the control room pushing the big red button or even doing the actual terrorist "hunting". They provide a platform that lets the people doing the analyzing/hunting to quickly and efficiently process data they already have. As the article points out, Palantir didn't add data to the system, it just made searching it and sorting it more efficient than having to log into 3 separate systems and search them out.
The company itself is essentially an elite technology consulting firm (albeit one that is actually technically competent). They have a set of core generic products (they have a finance analysis product, government analysis product, some other ones). BWStearn's description is accurate (a tool for analyzing entities and their relationships). But the key to their success is they take that generic product and then dispatch teams of engineers to customize each deployment for each client (hence consulting). Often clients will hire them because the client has engineering talent and has to hire Palantir because of that.
From what I've noticed the culture does learn fratty, but I didn't find it uncomfortably fratty. However, the people there work hard and they work long hours.
At least one of the non-clone ones mentions a strong Palantir internal push to pad GD with positive reviews.
I have no experience of their software, but the people I know who work there are very smart. Some of the best developers I've ever known. OTOH, many of the other best developers I've known refuse to work there on principle.