Why do you call this a failure?
This is "AI" parroting humans who made authorised commitments.
If you don't want commitments out, don't feed them in.
This is "AI" parroting humans who made authorised commitments.
If you don't want commitments out, don't feed them in.
The failure is architectural: once AI is allowed to draft at scale, “don’t feed it commitments” stops being a reliable control. Those patterns exist everywhere in historical data and live context.
At that point the question isn’t training, it’s where you draw the enforcement boundary for irreversible outcomes.
That’s the layer I’m testing.
Also I think confining irreversible outcomes to the results of commitments is unsafe. Consider the irreversible outcome of advice that leads to customer quitting. There isn't a separate "layer" here.
Training governs what a model tends to say. Authority governs what is allowed to be acted on.
You can’t pre-block bad advice, but you can pre-block unapproved financial or contractual actions.
That’s the scope.
"AI-generated messages making commitments no one explicitly approved. Refunds implied. Discounts promised. Renewals renegotiated."
And if you can't block bad advice, you have a bigger problem since you cannot block resultant contractual action by the recipient e.g. termination.