As unreasonable as that sounds, do you think it is that unlikely? Consider that developers and systems administrators make up only a tiny fraction of all computer users. At the same time, we are the ones most likely to use tools like ssh. Very few run-of-the-mill computer users would have a call to do so.

Also consider that developers and systems administrators (and in general the sorts of people who use ssh) are far more likely to have the technical know-how to launch a sophisticated attack.

Putting all that together, it would not surprise me if the use of ssh were used as an identifying marker by the United States government to single individuals out for increased surveillance. Grant you, if this happens I think it is absolute bullshit, but I think there is some paranoid bureaucratic logic that leads to this conclusion.