How so? And, what is a better metric, and why?
The Mahalanobis distance is a measure of the distance between a point P and a distribution D, introduced by P. C. Mahalanobis in 1936. It is a multi-dimensional generalization of the idea of measuring how many standard deviations away P is from the mean of D. This distance is zero for P at the mean of D and grows as P moves away from the mean along each principal component axis. If each of these axes is re-scaled to have unit variance, then the Mahalanobis distance corresponds to standard Euclidean distance in the transformed space. The Mahalanobis distance is thus unitless, scale-invariant, and takes into account the correlations of the data set.
Euclidian distance measures the distance between two points, while Mahalanobis measures the distance between a distribution (canonically multivariate normal) and a point. Mahalanobis distance is not a generalization if Euclidian distance, it's an altogether different concept of distance that doesn't even make sense without talking about a distribution with mean and covariance matrix.
Like Mahalanobis distance, apple pie is not a fruit and is not a generalization of an apple.
> Euclidean distance measures the distance between two points, while Mahalanobis measures the distance between a distribution (canonically multivariate normal) and a point
In a discussion about metric and metric spaces we dont care about those things, its abstracted out and considered irrelevant. All that matters is that we have a set of 'things' and a distance between pairs of such things that satisfies the properties of being a distance (more precisely, properties of being a metric).
@CrazyStat (I cannot respond to your comment so leaving it here)
I think you overlooked
> things that satisfies the properties of being a distance (more precisely, properties of being a metric).
that I wrote. Of course it has to satisfy the properties of being a metric. The red herring, as far as dimensionality is concerned, is the complaint that Mahalanobis is defined over distributions while Euclidean is over points.
The part about MD that you get absolutely right is its nothing but Euclidean distance in a space that has been transformed by a linear transformation. MD (the version with sqrt applied) and ED aren't that different, especially so in the context of dimensionality
@CrazyStat response to second comment.
It indeed isnt, its just Euclidean distance under linear transformation. I was just quoting you, you had said
> while Mahalanobis measures the distance between a distribution
My point was even it is defined for distributions its not really relevant.
> Mahalanobis "distance" is more closely related to a likelihood function than to a true distance function.
That's a subjective claim, and open to personal interpretation. Mathematically MD is indeed a metric (equivalently a distance) and it does show up in the log likelihood function. Mahalanobis was a statistician, but MD is a bonafide distance in any finite dimensional linear space, with possible extensions to infinite dimensional spaces by way of a positive definite kernel function (or equivalently, the covariance function of a Gaussian process)
This is not a red herring, it's a fundamental issue.
Mahalanobis "distance" is more closely related to a likelihood function than to a true distance function.
In short, much that I love Mahalanobis distances' many properties it does zilch for dimensionality.
https://stats.stackexchange.com/questions/99171/why-is-eucli...