It is the traveler's responsibility to know what activities are permissible given their visa, and to only engage in those activities. That's no different from everyone's general responsibility to abide by the law.

A US visa is simply permission to present yourself at a port of entry for admission, at which time you may questioned further by border control, and in rare cases denied entry.

Most B visas are valid for multiple entries over 10 years. The fact that you wrote a letter and brought it to your original visa interview (which may have been years ago, and likely wasn't even looked at by the officer, who in a busy consulate has less than 2 minutes to complete your interview) does not mean the US government has affirmatively granted you permisison to do everything on that letter.