Couldn't that just, never get an answer?
Or maybe just says "i don't know" with full certainty.
Or maybe just says "i don't know" with full certainty.
If that's not the case then it might just trigger bad risk compensation behavior in the model's human operators.