> anything you say to an officer of the law might be misremembered later and there is nothing you can do against that.
No.
Anything you say can and will be used against you in court. If you lie about anything, even something trivial, that is a federal offense and you can go to jail for that. And many people have.
It has nothing to do with mis-remembering. You have no idea why they're talking to you. It might be about someone else. It might be about you. But you can bet it's about someone or something. And something you say may incriminate either yourself or someone else.
> On the other hand, in cases like this, it would seem rude and disproportionate to not even meet these fine fellows from the FBI.
This is foolish. They're not there to pop in and say hello. They're doing their job and cooperating with that work is not in your self interest.
> How often is law enforcement confronted with people that don't talk to them on general principle?
Rarely. Most people are scared and don't shut up. However, when they do, it's almost always a lawyer and that assumption will work in your favor.
> Does that really make you suspicious or is it quite common?
It doesn't make you suspicious. See the above. Don't be rude or curt. Be kind, they are human after all. Just explain that you are exercising your rights and wish them a good day.