No, not really. The word "user" starts with a consonant sound, for example, even though "u" is a written vowel.
From what I have seen, people writing about linguistics do not usually bother specifying this distinction in this context. It is usually implied that an expression like "following a vowel" or "following a consonant" has nothing to do with orthography, but the actual phonetics. The orthography is not really considered a source of truth.
They are sounds as well as groups of letters, but the OP didn't specify, hence my clarification.
Some examples:
* "He has an MBA" because "M" sounds like it starts with a vowel (here: "em").