You're missing that a large number of people don't go into it "trying to break it"
Replace "GPT-3" with "Hacker News posters" "Wikipedia", or "News broadcasts" to create three more 100%-accurate paragraphs.
Even with the most credible news sources, it is still up to the person reading it to sense the BS.
It wouldn't be used to fly plane without lots of physical buttons as a fallback.
Composing rigid instructions for computer is already hard, even with precise semantics defined. Even with static typed, dynamic typed, developers will try hard to get rid of a single bug.
AI will serve as a middleware with an objective of arbitrary result.
Human
|> UI (request)
|> AI
|> UI (response)
|> Human
|> UI (request with heuristic)
|> Computer does thingNot today. Not yet.