Hacker News Reader

Comment by TrackerFF | Hacker News Reader

We received a presentation/demo of their products when I worked in the gov (not the US gov. though) a couple of years ago.

They seemed, okay? I mean nothing seemed mind-blowing. I worked on surveillance in a specific sector, where interagency collaboration is important. Hence why Palantir pitched their tools.

I'm not sure how they've managed to blow up like that. Do they have some extensive network with gov. officials, in the same way top management consulting firms operate?

EDIT: Basically their pitch was that if agency A and B (and C, D, etc.) connected their data sources to the tool (I think it was Gotham), then identifying and catching threat actors would be much easier, and that their software would streamline this.

View on HN

Ex FDE here. Part of Palantir's pitch for Foundry back in the day is connecting disparate databases from agencies that don't collaborate well. They're also really good at getting demos up and running in a few days. The tech was good back when I was employed.

They get hate because support war efforts / police / intelligence.

They do 5 year contracts with the govt then bump the prices once they're sticky, like a J curve, hence the valuation.

Yes. In practice, Palantir enables obfuscation of the "fruit of the poisonous tree".

Enabling a parallel construction in a surveillance state should draw hate, as it weakens the fourth amendment.

The thing is, Palantir doesn't enable that, their software does. And if Palantir didn't exist, the government could simply create it. (Some will say that the government is too incompetent to write software, to which I point to the US Digital Service, and also point out that trusting hyperbole is always a bad idea).

My point isn't that you shouldn't hate on Palantir, but that in doing so often the cause and effect get reversed in people's minds. They think that Palantir is the cause and eroded civil liberties is the result, when the opposite is true.

> They get hate because support war efforts / police / intelligence.

Right. That's exactly why they get hate. So what's the problem with that?

Sounds like the poster is saying we should also hate them for scamming everyone out of their country/state/municipality’s public money by jacking up the price of contracts once they hook them in?

OK, agreed, there's at least two good reasons.

Oh, they also bought thw VP of the USA. no biggie, just fascism.

Hate?

True, but government programs get cancelled all the time.

I am not sure what government you are thinking of, but not in the US government. There are some trimmings and some consolidation that may shift people and function in or under something else, but the programs, as generally defined by activities (because the nature of a “program” often morphs over time, even if everything else remains the same) basically never really get shut down anymore, if ever.

There may be a loss of interest or focus, often related to technological changes causing an atrophy over time, but that usually also just leads to long tail existence that seems to mostly be defined by the duration of the career length of those involved and/or any kind of special interests that can maintain their nepotistic/corrupt advocacy.

It is a function of the empire and the reserve currency status of it, i.e., the “infinite” money printing that causes profligacy that is also an existential threat because it distorts perceptions and reality, e.g., the belief that it does not matter and that no discipline in spending or budgeting is necessary, hence why America has essentially for 56 years now (ignoring the anomaly in 2001) not produced a balanced budget.

During the Apollo project years, NASA's budget was over 4% of the federal spend. Now, on percentage basis, NASA's budget is 90% less than it was. Defense programs and spending is wildly variable over time.

Didn't trump/doge just shut a ton of them?

Nothing that spends a lot of money on the scale of the Fed. Just low cost culture war stuff.

That's what gets attention in the media, but the Trump administration has also been axing a lot of "traditional" programs.

I used to work a defense contractor and still have some friends in that industry. One of them told me their project (related to some armored vehicle) was unceremoniously killed overnight a few months ago. As best they can tell it had nothing to do with cost, schedule, missed deliveries, etc -- it was just someone in the administration deciding they don't want to keep paying for it. They work for a very large contractor and apparently this happened to a whole bunch of projects / teams there.

> Do they have some extensive network with gov. officials, in the same way top management consulting firms operate?

Yes, in that one of their cofounders and his buddies bankrolled the current administration...

This was an article that really helped me understand it

https://sarahconstantin.substack.com/p/the-great-data-integr...

That's a great summary, especially the part about Palantir's strategic customer approach being the middle squeeze (high + low buy-in being used to override middle manager blockers).

The problem as enunciated also 100% tracks with my time as a consultant -- delivering solutions was 40% people-fixing and 60% code-fixing.

I've heard people say it's basically SAP with some spooky mystique for marketing.

Wake me up when SAP provides productivity tools of the level of PalantirFoundry’s Pipeline Builder, Ontology Manager, Data Lineage and Workshop.

I'm really glad I don't know what any of those things are.

They are the tools why PalantirFoundry is worth every penny

[not entirely true note: PalantirFoundry is a lowCode/noCode stack on top of a Spark cluster]

Pipeline Builder : https://youtu.be/WIQA7u1tbsY?si=iG-N9xHpUPPVAqDX

OntologyManager: https://youtu.be/SOW0IA_I0bk?si=u5eha0o-yrn4pHvu

Workshop: https://youtu.be/Uh0zpMUR6wY?si=BClTTbch70ZolRCb

I am sorry but these look like very basic ETL/UI Dashboard tools. Is this all they got?

I would love to know an alternative to that superbly integrated stack . [be aware that none of those tools require coding knowledge, nor any DB administration, nor scalability consideration. But for me the most important part is that the data workflow is not dataFrame driven, but semantic entity/relationship driven. Think of that as if you had an ORM magically wrapping your Spark cluster, and make it look like a graph database. Oh, and the sync with ElaticSearch is built-in for a no-brainer semantic search feature on your objects]

> be aware that none of those tools require coding knowledge, nor any DB administration, nor scalability consideration.

These are slogans with marketing grade truth, not engineering truth. The moment you leave the happy path wizards you run straight into Python, SQL, JVM knobs, and cluster limits. The visual Pipeline Builder drops you into a Code node the second you need anything beyond basic joins/filters. Here is an example: https://www.palantir.com/docs/foundry/code-workbook/getting-...

>> nor scalability consideration. Foundry Spark jobs expose tunable variables, you are expected to tweak them for performance and cost: https://www.palantir.com/docs/foundry/optimizing-pipelines/s...

99% of our pipelineBuilders are purely based on the available building blocks [that cover most of PySpark functions: https://www.palantir.com/docs/foundry/pipeline-builder/funct...]. Most notably we do join/filters/explode/group, Dozens of them, realigning data from different parts of the company.

Having most of those PySpark functions exposed visually as building blocks with debug points available at each step simply outperforms a codeWorkbook in term of maintainability and readability.

Yeah as a general rule, when something requires "no coding" I question if they're too zoomed-in. In reality, you can just hire a programmer instead of a box-clicker.

...Except in this case, there might be a lot of value in letting James Bond or Boris Badenov whip up a quick spy pipeline without putting up a job ad for a Python coder.

Once you have used a visual block-based version of PySpark, going back to regular Python code will really look like going back to prehistory.

> but semantic entity/relationship driven

This is marketing speak who means nothing. What do you mean? Something like this?: https://www.ibm.com/products/information-governance-catalog

Your data in Foundry are available both as tabular content and as a graph at the same time.

Here is a tool to visually explore your graph:

https://www.palantir.com/docs/foundry/vertex/graphs-explore

And here is the search engine to search for your objects (free-text and/or types): https://www.palantir.com/docs/foundry/object-explorer/gettin...

And here is the tool to manage the ontology (the types and relationships of the graph, plus the mapping of the graph to the tabular datasets):

https://www.palantir.com/docs/foundry/ontology-manager/overv...

The UI designer based on the graph data is called Workshop:

https://www.palantir.com/docs/foundry/workshop/overview/

I still have to find a stack where this duality is as integrated as in Foundry.

> I would love to know an alternative to that superbly integrated stack

This one has 10 years of precedence: https://www.ibm.com/products/information-server

Ontologies, semantic relationships, data pipelines, data provenance (I suppose a.k.a. lineage), etc. are all things that quickly become familiar to anyone working on knowledge management.

All of these things are pretty fascinating, as they concern the meta of the meta; the nature of knowledge itself; how to slice the ineffable reality into parts and how to communicate complex things to people.

Many software engineers are happy not knowing any of those terms. This perceived boredom is part of the reason why companies, some indeed with questionable ethics, like Palantir (is there even more than one in this space?) have very little competition and are making big bucks on big contracts (and only stand to make more once ML becomes less chaotic and more structured).

I definitely agree.

Two comments here:

- the real power of Foundry is that it is REALLY scalable. Palantir is selling all the knowledge management+data management that you mentionned BUT at a level of integration and scalability that I had never seen in any other product.

- someone in the discussions mentionned IBM InformationServer as an alternative. Functionnaly that sounds quite similar to Foundry. The question is whether the implementation choices are as clever as in Foundry. #toBeInvestigated

> how to slice the ineffable reality into parts and how to communicate complex things to people

So, lots and lots of pie charts?

Slicing reality doesn’t make it into a pie!

You are aware the product names are cognitive obfuscation that the underlying implementation is just vanilla open source data modeling services right?

Linux. Windows. Both are "operating systems"?

What actual novel data collating and anomaly detection algorithms does Palantir implement that no one else does? Oh that's right these services are all just different corporate branding atop the same old open source stacks and libraries everyone uses.

I doubt anyone has the $$$ horsepower to maintain a coherent stack as big as Foundry. So in the makeOrBuy, you probably will go for a buy. And in that field, I still think SAP is way behind Palantir. Someone in the discussion mentionned IBM’s InformationServer, and I will spend some time to investigate its storage scalability strategy and the data model governance.

At least Astronomer has Gwyneth Paltrow

Show me a fully graphical version controlled UI that hides PySpark behind visual widgets.

Show me a entity-relationship model management [with version control] that wraps Spark data frames into a fully ontology-based graph database.

Show me a low code/no code UI designer that is fully consuming the ontology layer abstracting your Spark dataFrames.

#meWaiting

Yeah ok let me go expropriate other peoples systems.

They exist but are used in house not sold as products.

There are information theory texts with all these ideas written in them published back in the 1950s we just didn't have powerful enough computers

Thiel and Musk's only advantage is age. They were able to implement first due to a single property of physics.

Most "geniuses" today, like Tao, just like reading math. None of them have any truly novel net new ideas. They're parrots whose cages were in the nerdy section of a book store.

This is like the equivalent of "my girlfriend goes to another school, you wouldn't know her". Okay, so there's in-house systems as good as this. Big whoop if no one can actually see or use them.

I still wait for my list of immediately available products, so my company can escape the evilness [and overefficiency] of the PalantirFoundry stack.

Pretty sure Tao’s had one or two new ideas.

I’d like to hear from others, but my assumption has been that they’re the ones that 1) have staff with the required clearance to work on DoD projects and 2) the required security and compliance certifications in their product. On the latter, it’s not easy to provide a product that is DoD IL5 certified, so that is a differentiator for them.

This is the actual answer

I was in the IC when Palantir was being rolled out and took a bunch of their training courses at their facilities in Georgetown

They have the data storage that complies with DCID and RMF ATO requirements across every IL and compartment

Before Palantir the only thing that we had was Analyst Notebook and you had to have a CD to run it and manage your own data repos locally

Palantir was entirely browser based and you didn’t have to manage data at all so they killed AN almost immediately

since i was not sure whats included in IL5, i asked GPT: >> FedRAMP Moderate baseline (as a foundation)

Additional DoD SRG IL5 controls on top of FedRAMP

Physical and logical separation of IL5 workloads from lower-level workloads

U.S. citizenship and background checks for all cloud personnel with access

Hosting and data storage within U.S. territory

Continuous monitoring and incident response plans that meet DoD requirements

DoD-specific access controls and encryption <<<

Thats quite some strict list which most companies would just not pass...

just speculating, but part of the "nothing special" piece may be that the best engineers/software devs who dream of "changing the world" for the better won't give them the time of day, but they can do "just good enough for government work" analytics / software dev for government work

> Do they have some extensive network with gov. officials, in the same way top management consulting firms operate?

It's a little bit of being good at sales but it's also very much that the integrations with their tools and platforms are synergistic to a large degree.

> Do they have some extensive network with gov. officials, in the same way top management consulting firms operate?

Don't known about any other government officials, but Austrians ex-chancellor Sebastian Kurz has strong ties with his Tel Aviv-based cybersecurity startup Dream[0].

[0]: https://www.politico.eu/article/austria-former-chancellor-se...

Why would their pitch to prospective targets reflect their corporate aims?

Do facebook or google do this to their users?

They're blowing up because they've got allies in government right now steering them contracts, which is par for the course for big government contracting. Elections are to a certain extent a contest between different camps to swing at the Piñata of Federal funding, with the winner taking the current batch of candy.

I kinda think Palantir is GenZ TRW, or at least the data analytics sections of TRW that did things like the first US social credit... I mean... credit score systems:

https://en.wikipedia.org/wiki/TRW_Inc.

SAP is also a good comparison, so maybe GenZ SAP.

There's a ton of conspiracy theories about them because their founders are tech-right ideologues. I'd pretty much guarantee that if we'd elected the most mega-woke president in US history instead of Trump and they wanted to commission a huge government data project to analyze the entire American workforce to look for racial, ethic, and sexual orientation biases, Palantir would bid on that contract.

Not saying they're doing great stuff, but the bad stuff they're doing is the fault of your elected representatives. People voted for this.

> because they've got allies in government right now steering them contracts

Is that because palantir takes them out for golf outings and steaks? Or is it because palantir promises certain results and outcomes to them once the product is in place?

> There's a ton of conspiracy theories about them because their founders are tech-right ideologues.

Yea, it's just the ideology of their founders, it has _nothing_ to do with their actions and partnerships.

> I'd pretty much guarantee that if we'd elected the most mega-woke president in US history instead of Trump and they wanted to commission a huge government data project to analyze the entire American workforce to look for racial, ethic, and sexual orientation biases, Palantir would bid on that contract.

"They're evil and they don't care who's buying."

> but the bad stuff they're doing is the fault of your elected representatives.

That's not how morality, ethics, or the law works.

> People voted for this.

Literally _no one_ voted for this. The people who wanted DOGE to clean up government spending did not in any way think that all federal data, including their own private medical data, would be, or even need to be, fed into a private company to be used in service of the problem.

This is ridiculous apologia for evil. What is wrong with this site? If there's even a single dollar of profit on the table then civil rights are the first thing the "hacker news types" sacrifice. It makes you all seem gross and unapproachable.

My intent was to express skepticism about the idea that Palantir alone is anything other than a contractor behaving like a contractor. They’ll bid on anything and so will any other contractor. I get skeptical when something like that is held up as a big bogey man. I think it’s a distraction.

The real issue is that people voted for this. All of it.

Anyone who voted for Trump after J6 knew what they were getting. All the people with buyers remorse about Trump make me think of this parable:

https://www.northerncherokeenation.com/boy-and-the-rattlesna...

“You knew what I was when you picked me up.”

Reply on news.ycombinator.com