If you're a woman, yes, it's probably true. Like 10,000,000:1 chance.
If the boss has known me for a long time and it seems completely out of character for me (which it would be), then maybe he talks to me about it to see WTF is up.
But some dude who just got hired? Assuming there's no record and no witnesses? You have to go with the vastly more likely case that he did it.
If that gets 1 in 10,000,000 guys unfairly fired, but gets 9,999,999 guys justly fired, that's a pretty good outcome in my book.