> but I very much doubt that, because I didn't authorize my security key to log into this server, never mind any other afterwards.
What? This server pulls your public key from github and allows your client to connect to it. If you have forwarding turned on this server can INDEED connect to every server you are connected to.