If I remember,
SSH server don't share the public key it accepts.
Your SSH agent will try all your key one by one (you can change this behaviour with custom config)
Basically, don't reuse keys in places where you might not want to be identified and use ssh configs to prevent announcing all keys to the world.