The main reason is a (maybe not on HN) little known scenario where the user has SSH key forwarding enabled, and the host they SSH to takes that forwarded identity and uses it to, say, fetch your private repos on github and the like.
https://news.ycombinator.com/item?id=9425805
https://www.reddit.com/r/netsec/comments/3frnxb/my_ssh_serve...
I'd have _no problems at all_ signing a pgp message the likes of "I'm proving I am myself, and that you requested me to sign also with token QIMdoV76LIvymGvTxXEB8LkIIqfM4nEm5W"