ssh-add has a -c option, which uses ssh-askpass for confirmation. Unfortunately it doesn't give much information, but you hope you would notice it coming up when you didn't initiate it. You may also be able to do something with gpg-agent, it seems to have more options.