The way people "follow" you requires you to add them as a contact. Your contact card is basically a public key, but in a more human-understandable form.