So let's go through a hypothetical situation:
A has friend 1 and 2. He vets them, checks their ID, goes to a key-signing party, trusts that they are 1 and 2 with his life.
Now he gets an email from "5000" who claims that he's from the IRS and needs to know information from you.
You look up this "5000"s signature.
Now, none of A's friends work in the IRS, but "5000" was signed by "500","302","201" and another 500 people.
"302" was signed by "23", "23" was signed by "2".
Should "A" send him the info?
Theoretically, he can call "2", ask him how "2" knows "23", then call "23" on behalf of "2", and so on.
But he doesn't know if "23" is honest, and all the more so "302".
And what if "302" says that he knows who he is because he self-identified as "5000 from the IRS" at a bar?
Too much thinking.
Now imagine A is the kind of person who downloads EXEs to view cat pictures?