You watch too much TV. I know it's hard, but when talking about real life law enforcement, try to forget everything you've ever seen about it on TV. It's about as accurate as the hacking scenes in NCIS.
The justice system in the US is completely broken. You should be very afraid.
Who knows... you do hear of cases where crimes are pinned on innocent people, seemingly because some DA would rather prosecute anybody than leave it 'unsolved'.
I called the police, they interviewed me. Came back a few days later and did it again. At the time, what I remember feeling like was that they were trying to figure out if I made it up and kept the money rather than actually catching the person who had a gun to my head. I was pissed.
OTOH, I think the advice is strongest in the case where the police are initiating contact, and you don't have a pre-existing and independent basis to think there is an interest in talking without delay that justifies the additional risk of doing so without an attorney.
The question is, what are the odds? Most people walking through the door don't do that to me.
But if the police come to your door, what are the odds that they're going to treat you unfairly? I suspect, higher than the odds that the person I hold the door for is going to punch me. Are the odds with the police unacceptably high? For me, no. (But then, first, I've never yet been treated unfairly by the police, whereas at least three times I've received better treatment than I objectively deserved. And second, I'm white and fairly conservative in appearance, so I'm probably going to get as much benefit of the doubt as anyone ever does.)
What are the highly likely penalties for an assailant attacking you in broad daylight like that, in a society under rule of law?
Now, what are the penalties to the FBI treating you unfairly during a surprise visit?
Further-- in the extremely unlikely event of getting punched in the face in broad daylight because you held the door open for somebody, what's it like to recover? Keep in mind that the public at large, law enforcement, and the court system would very likely all be on your side as you seek medical care, justice and remuneration.
Same question for FBI treating you unfairly after entertain their surprise visit without counsel. What does a recovery from that look like, and who would you likely have on your side?