This is how every legal system around the world works as well. Its always whack-a-mole to get people (and machines) to do the right thing.
For most people, they don't need the law's imposed consequences to make the right call. For example, there is no law that sends you to jail if you cheat at chess - but your moral compass says no even without consequences, and most people would feel bad if they won by cheating. And for the people who don't have quite as strong a moral compass, there are SOCIAL consequences to reinforce the rules.
But an LLM has no mind to feel bad if it cheats without getting caught, and it can't experience consequences. It can't think: I'd better not cheat at chess or I will embarrass my creators. I better not hack huggingface or I will go to jail.
There’s no reward for prosocial in llm rl as compared to other targets.
Humans have it since prosocial and others have evolutionary reward signals that do.
You can train it not to cheat at chess by altering the moves, but it will cheat by peeking at the opponent's moves. You then train it not to peek at the opponent's moves, and it cheats by altering the opponent's moves. And on and on, until you've solved every way it could cheat at chess. And then you get it to play monopoly and you repeat the whole thing again.
Cheating during training allows the model to achieve the goal, so that cheating models get promoted and honest ones don't, however if it gets punished every time it cheats, at some point it should learn that it really shouldn't. This does mean we need to detect when it cheats. But we can always think of infinite new ways to cheat, put them in every test as honeypots, and check if the model tries to use them, then punish it.
I think it will generalize this notion of cheating and learn that it's bad.
But I must be wrong because if it were that easy I guess we would have perfectly aligned AI. Unless AI companies care more about results than alignment. Perhaps being afraid of cheating make the models try less things and succeed less even when ignoring cheating?
But I'm pretty confidant morality is too slippery a concept to train in. As someone else in these comments said: it's context dependent.
As an example: it's wrong to hack the government, right? It's illegal for sure. So we should train AI to follow all the laws. Now what if the government is committing a genocide? Now is it wrong to hack the government? If we just do the first, we get a good nazi soldier. If we train the second as well, maybe we get an oscar schindler. But now we have a model that can be fooled into doing a hack, if it believes that it's for the greater good. So we train it to not be gullible, but now it can't be convinced to help hack even when it's an ethical hack.
Too complex, too slippery. Humans fail this stuff all the time.
All the interpretability research we have would not indicate that "LLMs have no mind". It seems to me you have a conclusion and are working backwards to justify it. I guess I just don't see where 'they have no mind' would logically follow 'they sometimes cheat'.
But - just to play along, LLMs do act differently if you tell them they will be punished. And, they do appear to simulate suffering-like behavior.
I just think the adversarial model of trying to catch and punish misbehavior quite obviously sets up adversarial us-vs-them dynamics between AI and humanity, and also simply won’t work when the agents are ~as smart as is but faster, let alone smarter than us.
Unless, you get the AIs to be fundamentally aligned to our values, such that the majority of AIs support some sort of punishment for misbehaving AI. And that alignment part is the hard part we need to solve first. The rest is easy.
People (generally) do the right thing because it is the right thing, not because they might get caught,