ChatGPT intentionally plows through low-confidence scenarios. It just tries to do its best because "I dunno" is reliably low-utility while a hallucination doesn't gamble lives. It errs on the side of hallucination by design: if you alter the RLHF feedback, you can alter the confidence level.

In FSD, as you point out, the stakes are higher. It makes a different design decision: it slows down in low-confidence situations. Just like a person. If you watch a FSD video, you'll see that most of the human interventions happen because FSD lost confidence in a tricky scenario and halted the car even though the pathing ultimately wound up being correct. This won't solve every edge case, but neither do humans -- the bar is (or should be) "better than human," not "perfect," otherwise we are sacrificing lives on principle.

I have no idea if training will stall out half way through the march of 9s. It might, but if it does, it won't be because there is no high level strategy to control hallucinations. That already exists.