The goals for a chatbot assistant are to be useful, correct, and not insult people. The goals for a defense AI are to extract correct features, provide useful guidance, and not kill the wrong people. If you are working in defense you already have a belief that your work is morally correct: most of those justifications are either that your work will kill bad people more effectively, and so save friendly lives, or will pick who to kill most correctly, and so save innocent lives. Having an AI that is better aligned towards those goals are better.
You may disagree that working in defense is ever morally justified! But Palantir dont't share those beliefs, and want to do as good of a job as they can, and so want the most aligned AI model they can.
Collateral damage. Same thing that happens in any war when an analyst or soldier misreads the battlespace.
War is hell. We won’t change that by making it pleasant. We can only avoid it by not going to war.
https://www.theguardian.com/science/the-lay-scientist/2016/f...
So what will happen if AI suggests someone is a terrorist when they are not? Well, in the worst scenario they'll be killed, and it'll be very close to what we have today, except somewhere in a private military database there might be an automatically generated record from an LLM ok-ing the target.
Or you don't care about morals. Or you are evil.
Which, when you unpack it, is even more interesting. If you do embrace the emotional aspect of war you end up with situations like the my lai massacre. Does AI have the ability to prevent war crimes while engaging in "legal" killings feels like an interesting philosophical question.
Liability and regulatory scrutiny are factors. They’re liable about offensive speech but military use cases are an effective shield against liability given that deaths are expected.
there's nothing preventing them from implementing a toggleable filter, with a big red warning, geo-restrictions and even age verification if you want to turn it off.
it's not legal liability that prevents them from doing that, it's ideology.
Which would not placate regulators in all likelihood. Close regulatory scrutiny is a big problem for companies seeking to grow, and a pesky cost for big companies.
Jokes aside; All frontier model AI companies were closely associated with US big tech companies that were facing threats about breakup from the Democrats (OpenAI/Microsoft, Anthropic/AWS, and Google). They really didn't want AI anxiety to be one more reason they were disliked by Washington.
Same reason we don’t excuse bad manners because someone is a soldier. It’s easy to be polite, and the damage is done at home. It’s harder to accept subordination to a foreign power that pursued an obvious military direction. And the damage of your own system will hopefully happen elsewhere.
This hasn’t been true in a long time. (Some countries took longer to learn it than others.)
the Obama administration did this with the drone program...10 years before ChatGPT or LLMs
Who knows tomorrow we might be the enemy.
https://www.tomshardware.com/tech-industry/artificial-intell...
The only reason they are loudly announcing "Hey our military is using YOUR open source models" is to trigger fearmongering and overzealous regulation in the West and make us less competitive in AI.
This was used by Tencent for their SOTA open source model (Hunyuan).
Sam probably wants that automation to be very robust to abuse, and avoid entangling the company in any sort of nonsense -- legal, cultural, or otherwise. Do the job, do it well, and stay on script. Don't insult the customer. Take abuse with a smile in your face. You know, the sort of stuff that the human people doing those roles now are trained on and understand.
To the extent that Sam can get woke-y extraordinarily cheap faculty/phd students to do enthusiastic labor for his customer service automation by calling it "alignment" or "AI Safety", well, all the better!
Not sure what any of that has to do with whether Sam or his company supports the use of automated weaponry.
As for how faculty and students characterize this type of research? There are a few kinds o things going on, not least of which is ego. But the most important, from Sam's perspective, is "academia-washing". But using a university as his contractor and calling his contractors' junior employees "students", Sam gets to skirt the whole visa thing!!! This for the small price of a faculty member "academia-washing" his internal R&D problem statement.