Sorry - people misappropriating math jargon into the mainstream is a little pet peeve of mine. I agree with your main point though.
It means at least:
1. having a zero inner product
2. (a square matrix) that is the inverse of its transpose
3. linear transformation that preserves angles
4. statistically independent
But you are correct, sir ;-)
Used to describe two things that are independent of each other. One does not imply the other.
"Common sense and intelligence are orthogonal. I've seen plenty of smart people with no common sense."
In IT, I've heard the phrase "mutually orthogonal requirements" for years now to indicate requirements that do not overlap.
In mathematics, orthogonal means perpendicular in a geometric-sense (think two vectors) but can also be used in other contexts with a different meaning.
This is why the word "perpenidcular" is not used, as sometimes your vectors don't really have "directions" in the intuitive sense of the word (e.g. the inner product space of functions).
When you decompose a goal into several linearly independent sub-goals - those sub-goals are said to be 'orthogonal' since they don't have any interaction/interdependence with one another.