In the long run, perhaps the most dangerous aspect of LLM tech is how much better it is at faking a layer of metadata which humans automatically interpret as trustworthiness.
"It told me that cavemen hunted dinosaurs, but it said so in a very articulate and kind way, and I don't see why the machine would have a reason to lie about that."
1. It takes longer to verify/debug/fix specious results than to just do it manually.
2. Specious results were not reliably checked, leading to something exploding in a very bad way.
Since they don't actually model facts or contradictions, adding prompt-text like "provide alternatives" is in effect more like "add weight to future tokens and words that correlate to what happened in documents where someone was asked to provide alternatives."
So the linguistic forms of cautious equivocation are easy to evoke, but reliably getting the logical content might be impossible.
But I wonder if tuning the output to avoid definitive statements would be beneficial from a UX perspective.
Heck, imagine how terrible the opposite would be: "When answering, be totally confident and assertive about your conclusions."