The reason the firms do not want to invest in making fact-checking a first-class feature is that the appearance of being right is what people want from AI.
The reason the firms do not want to invest in making fact-checking a first-class feature is that the appearance of being right is what people want from AI.
No, actually being right is what people want from AI, "the appearance of being right" is all that AI companies can deliver. It comes with the benefit that many people will be fooled into thinking that AI is more capable/useful than it actually is. AI companies have to either convince others that their product is something that it isn't, or that at least it will one day be something much more than it is.
You are considerably more optimistic than I am about how people use things like this. IMO people are happy with these tools if they can be used to support their existing positions and biases.
Even vibe-coding is like this. OK it creates code that compiles, but its primary job is still to confirm a bias. I have yet to see people making significant novel discoveries about functionality this way.
I wonder how popular it'd be if someone started an AI company that was explicitly advertised as "blowing smoke up your ass as a service", and clearly said that their chatbot was written to always agree with anything you said and to tell you whatever you want to hear. I don't doubt there's at least some market for it, but I'm guessing not many people ask for that in their prompts with current AI offerings.