I dunno man, I'm with the poster above (do we call that the gp?): We have lots of evidence that people in general don't have great truthiness filters for things they read online, and we have a tool that can very confidently say things that seem truthy but are not. Even before the internet, just in the age of radio and early TV, Churchill said "A lie gets halfway around the world before the truth has a chance to get its pants on."

Now we have a tool that isn't lying through malice, volition, or any sort of intent, and that will combine jargon and phrases that seem authoritative into its misinformation.

IMO responsible engineering involves considering the ups and downs of each solution - that's where the famous HA saying "if you have two you have one; if you have one you have none" comes from, dictating that we need at least 3 instances of a server for high-availability, right? What would it take to engineer this new tech for high accuracy? Where is anything other than the hubris of "we were so busy asking if we _could_, we never stopped to ask if we _should_" ?