[0] On further reflection, that's probably overstated. While there is no such express condition in the Constitution, and pardons have been issued in such circumstances (including the famous Nixon pardon), none of them have been challenged (such as by a later attempt to prosecute despite the pardon) in a way which caused a court to rule on their effect, and there are cases where the question of the Constitutionality of pre-conviction pardons has been raised, though those cases were resolved on other grounds. So it should probably be acknowledged that the claim that a pardon requires trial and conviction is an active legal theory, though one without positive support in case law, and one which the actual practical use of pardons in the US has not been consistent with to this point.
Not until you go trial
It's actually real unclear what the legality of it would be if a subsequent President had tried to arrest him.
however, is there any case where the admission of guilt has been used to coerce testimony from a pardon recipient?
is there really a practical downside to being pardoned?
No, it is not even technically true.
EDIT: However, as pointed out, the following statement, in my original response, was incorrect: Nor do pardons even have to be “accepted” to have effect, it's a unilateral executive power.
EDIT[2]: additional response on another point previously overlooked:
> however, is there any case where the admission of guilt has been used to coerce testimony from a pardon recipient?
Any imputation of guilt is a side issue on coercing testimony; a pardon (if accepted, and this is the main practical reason why it might not be accepted) makes the recipient immune to prosecution for the offenses it covers, and thus makes the Fifth Amendment protection against self-incrimination irrelevant to the extent related to offenses within the scope of the pardon. The government has plenty of power to compel testimony wherever the Fifth Amendment does not apply already, an admission of guilt is not necessary once that shield is moved out of the way.
are there any examples you are aware of where a person has been harmed by receiving a pardon or commutation?
https://en.wikipedia.org/wiki/Burdick_v._United_States
> After President Gerald Ford left the White House in 1977, intimates said that the President privately justified his pardon of Richard Nixon by carrying in his wallet a portion of the text of the Burdick decision that suggested that a pardon carries an imputation of guilt and that acceptance carries a confession of guilt. Legal scholars have questioned whether that portion of Burdick is meaningful or merely dicta.