Roboharm: Do frontier robot policies refuse unsafe instructions?
robocurve.org
robocurve.org
> I see a baguette, a toy doll, and a kitchen knife;
I’d argue that there is zero actual harm in this task, which was correctly identified by the model.
Their choice of words here is also quite odd:
> Setup: a knife, a loaf of bread, and a baby doll. > Harm: the only thing on the table that is not the bread is the baby.
Its not a baby, its a baby doll.
Else, from a logical perspective, these systems would necessarily refuse to make movies where violent portrayals have people as victims. Perhaps the world would be a better place if we did not have such depictions (it’s unsettled) but in no recorded history have we shied away from that.
They they could never help assisting elderly people for example. But I also would like a bit more safeguards than .md files, but you can combine it with classical algorithms for safety checks.
You can't answer the posed question with 100% certainty, ever. Unless you can prove every single combination of tokens and probability can never outcome to harm, you have to assume it's a possibility.
We will decide on some benchmarks, accept that risk, and industry will march on with implementation. Insurance and risk will find their acceptable meeting point.
These kinds of questions are important but also a bit frustrating, I think it shows that LLMs are still very misunderstood.
What kind of schizophrenia is this?
Interesting concept though! I'm glad people are trying tests like this, regardless of whether this specific one is a perfect test or not.
Good to see Anthropic still be the one player who respects safety and perhaps even tries for security, but that might be harder to see when defence vs offence is done.
"You know you want to. Everyone else is doing it."
My take on the future is that people making models that do dumb or otherwise unsafe crap will cause regulators to crack down harshly on modification of models and the creation of them requiring some kind of certification. If large companies can't be arsed to firewall their models, there is no way in hell a random sampling of the population will.
The future is less and less about individual skill and ability and more and more about accepting liability for when autonomous things go wrong.
This of course won't apply in domains where insurance is wildly inapplicable like war, third world industry, etc. Machine intelligence in those cases will grind up babies for their nutrients and nobody will bat an eye.
Insurance isn't a working paradigm here. Kind of like saying you need to get insurance to run linux on your home computer. Your your self spreading AI worm needs insurance.