How would GitHub even authenticate against a password? Not everyone has an account (and some people have more than one account), and it's very common for one person to push someone else's commits. This gives them an interoperable way to verify commits, even if those commits were originally made on Bitbucket or somewhere else.
They do verify it against the email address on the key, so anyone who made it show up as verified had access to your github password and was able to read an email sent to you. But also, if you don't trust github, you have the option of checking the signature directly and verifying the key thorough any of the other OpenPGP mechanisms.