It often does.
Not everyone is like you.
Not everyone wants the same things from a conversation as you.
I agree that a lot of times, the veracity of these details don't matter. I know I'm going to get a lot of hate for this, but: In my experience, there is a strong correlation between those who do not like being fact checked in these social situations, and those who make factually incorrect statements when it clearly matters (e.g. accusing someone of poor behavior, business transactions, etc).
We may be in the minority, but some of us do care to know whether you can be trusted. It's one thing to be up front and speak tentatively, and be open that you've gotten details wrong. It's another to give a fantastic narrative that happens to be full of incorrect details, and then complain when someone points it out.