Hacker News Reader

Comment by sfpotter | Hacker News Reader

The dot product isn’t a distance

My favorite kind of pedantry is pedantry that's naive - on like day 1 of topology 101 you learn that the "dot" product is an inner product and every inner product naturally induces a metric so every inner product space is a metric space (with the induced metric).

It’s not pedantry. Just because an inner product induced a metric space does it make it the same thing as that metric. The performance of similarity searches can depend very much on whether you use a dot product or a distance.

A distance is a function d(x, y) where d(x, y) = 0 if x = y, if x = 0, or if y = 0 (among other properties). If x = y and neither x = 0 nor y = 0, and if we let d(x, y) = x*y, then d(x, y) != 0 in general, so the dot product is certainly not a distance. Your comment isn't relevant, but thanks for calling me a pedant. ;-)

`is != induces`

Dot product is equivalent to cosine similarly for normalized vecotrs

Given normalized vectors, it's also proportional to the square of the Euclidian distance. Meaning for ranking (and thus k-nearest-neighbors) it doesn't matter if you use Euclidian distance or cosine similarity, you get the same result.

It isn't equivalent to it. See my other comment.

Dude.

Cos(x, y) = x . y / ||x|| * ||y||

If x and y are normalized:

Cos(x, y) = x . y

x*y/|x||y| is a very different function from x*y

It also isn't a distance...

It is for unit vectors.

My math is weak. I thought dot product measures alignment. Help me grok what "distance" means here?

Unit vectors have a normalized length, so the only difference between two unit vectors is the "heading". The angle between them can be interpreted as the the "difference" or distance, between them.

Since you can compose any vector in the space out of unit vectors, you can extend the concept. See above comment inner product -> metric space.

Hmm, I think I got my math wrong too.

What is a proper distance function is (1 - |a*b|). That's proportional to the distance between those points on the plane spanned along the circumference of the circle/sphere/hypersphere as it's projected onto that plane.

The dot product of two unit vectors is the cosine of the angle between them. Since cos^-1 (u*v) is the angle between them, or the geodesic distance between u and v regarded as points on the sphere, which is a distance. But the dot product is not.

If u = v, then u*v = 1, but for two points not equal to zero, the distance between them should always be zero. Remember that u*v = 0 if u and v are perpendicular. If neither u nor v are 0, then for them to be perpendicular, they must certainly not be equal. The dot product does not behave like a distance.

Reply on news.ycombinator.com