> What is the worst that can possibly happen as a result of AI being able to generate text that is indistinguishable from something a human wrote? Or that a "chat bot" can answer questions with more relevance and detail (correct or made up) ?

Remember the holodeck in Star Trek? Everyone thinks that's cool technology and not really particularly scary. But in reality, the existence of a holodeck is an existential threat to humanity: if you can have literally any experience you want in a holodeck, there is no reason to invent/do anything else.

AI text generation has a similar flavor of danger. Imagine a world in which everyone has a personalized better-than-human text-generating AI. People will have no incentive to read anything other than what it writes (why read Shakespeare when you could have Shakespeare-tailored-for-you?) People will have no incentive to broadcast their own words over those of their AI.

Obviously text is a small subset of the "literally any experience" offered by a holodeck, but it is not hard to see a future in which everyone is MITM'd by text-generating AIs.