Perhaps this helps: While fundamentally, in theory, they (keys and operations) are symmetric, many higher-level cryptographic protocols and their implementation do have differences (such as the private key also embedding the public key, or encryption being hybrid-symmetric). If the abstraction level is not obvious from context and you are still learning, this can be confusing.