> namely both "private" and "public" key are the same thing; the labeling "public" vs. "private" reflects your arbitrary choice of which of one you give away, and which one you keep for yourself.

It is not quite arbitrary. With RSA, you could potentially store only the modulus and the exponents, publishing one exponent and keeping the other one private (or making each privately known to different people, with both having the modulus). However, the way it is commonly stored is with the private key file includes both exponents and several other numbers, and the public exponent is usually 65537 which makes it easy to guess so you cannot effectively keep it secret. With some other kinds of cryptography (other than RSA), you can figure out the public key from the private key even without doing things like this.