Probably would’ve still been harassed and wound up on the no-fly list, but yes, if the FBI tricks you into meeting them, walk away.
If you are in such a situation you are already in trouble due to power imbalance. In other places where you can't get a lawyer / have friends high up you might as well play along...