All models that we train today have edge cases where the model produces wildly inaccurate output. As far as I'm aware, nobody yet has managed to squeeze this problem out.
ChatGPT is still an overall useful tool with the occasional hallucination. Those types of edge cases in software driving a car are unacceptable.
In FSD, as you point out, the stakes are higher. It makes a different design decision: it slows down in low-confidence situations. Just like a person. If you watch a FSD video, you'll see that most of the human interventions happen because FSD lost confidence in a tricky scenario and halted the car even though the pathing ultimately wound up being correct. This won't solve every edge case, but neither do humans -- the bar is (or should be) "better than human," not "perfect," otherwise we are sacrificing lives on principle.
I have no idea if training will stall out half way through the march of 9s. It might, but if it does, it won't be because there is no high level strategy to control hallucinations. That already exists.
Customers are willing to tolerate the latter right now because they believe the former is still in development.
I'm not here to argue that the overly enthusiastic timetables weren't cringe. They definitely were. I'm just here because there seems to be consensus that FSD is doomed and it doesn't remotely seem that way to me.
My point is that in the long run, Tesla cannot solve the ML edge cases by having a human available to take over at a split second notice.