Comment by srean | Hacker News Reader

Comment by srean | Hacker News Reader

Parent Full thread

In Math one encounters so many results that leave one with the impression that Squared Euclidean is special. One such example is Singular Value Decomposition, or equivalently the Eckart-Young theorem. Arithmetic mean also minimizes the sum of Squared Euclidean from a set of points. Squared-Euclidean's properties are also the reason why the K-means algorithm (Lloyd's algorithm) is so simple.

Note that the squared part is important in that result although the squaring destroys the metric property.

A part of beauty of Euclidean metric (now without the squaring) is it's symmetry properties. It's level set, the circle (sphere) is the most symmetric object.

This symmetry is also the reason why the circle does not change if one tilts the coordinates. The orientation of the level sets of the other metrics considered in the post, depend on the coordinate axes, they are not coordinate invariant.

Euclidean metric is also invariant under translation, rotation and reflection. It has a specific relation with notion of dot-product and orthogonality -- the Cauchy-Schwarz inequality.

A generalization of that is Holder's inequality that can be generalized beyond these Lp based metrics, to homogeneous sublinear 'distances' or levels sets that have some symmetry about the origin [0].

The Cartesian coordinate system is in some sense matched with the Euclidean metric. It would be fun to explore suitable coordinates for the other metrics and level sets, although I am not quite sure what that means.

[0] Unfortunately I couldn't find a convenient url. I thought Wikipedia had a demonstration of this result. Can't seem to find it.

Most of the special properties can be traced back to its special relationship with the inner product. And inner products have somewhat more elementary properties, so in that sense it explains the special position of the euclidean norm.

This has nothing to do with the coordinates by the way. If you want a different norm you'll first have to figure out an alternative to the bilinearity that gives the inner product its special properties.

Though bilinearity is pretty special itself, given the link between the tensor space and the linear algebra equivalent of currying.

> This has nothing to do with the coordinates by the way.

I think it does. Both decompose along orthogonal directions. See my comment here https://news.ycombinator.com/item?id=45248881

I think that's arguably an a posteriori explanation: you can find orthogonal coordinates with respect to which the L^2 norm has a nice form, but you can also single out the L^2 norm in various ways (for example, by its large symmetry group, or the fact that it obeys the parallelogram law—or even just the fact that "orthogonal" makes sense!) without ever directly referencing coordinates.

I mean they decompose in orthogonal components for all Lp norms I think? Is there a norm for which (x,0) is not the closest point to (x,y) on the x-axis?

Not quite sure what you mean but now I agree with your previous point that the coordinate system is irrelevant because these normal derived Lp metrics can be specified in a coordinate free way just by specifying their unit ball set

Wel what does it mean to be orthogonal? I'd argue that if you only have a metric then a line (segment) is orthogonal to some set if it is the shortest path from a point to a set. For norms it should work to only consider straight lines.

What I was getting at is that the x and y grid is orthogonal for all Lp norms. The interesting stuff happens if you pick a different line as x-axis. Orthogonal projection to a different axis still works though, the resulting grid just won't be orthogonal in the conventional sense.

I can't edit my comment anymore so let me elaborate a bit here.

What is this sneaky connection between squared Euclidean and Cartesian coordinates that I mentioned ? Why are they such a compatible pair ?

The answer is the Pythagorean theorem.

The squared Euclidean distances decomposes nicely along orthogonal (perpendicular) directions.

    d^2 = x^2 + y^2.

The Cartesian coordinates decomposes a point along orthogonal (perpendicular) axes as well, which we know is special for squared Euclidean distances.

The other metrics considered in the blog post decompose as, for lack of a better name, Fermat's last theorem decomposition.

    d^n = x^n + y^n

Now if we were to use a coordinate system that decomposes points like that, that would be interesting to explore. I don't know of coordinate systems that do that.

This much is true, forget about integral triples (lattice points) for integral n > 2.

I don't understand. The Cartesian coordinate system works fine in any norm. See the OP article.

The special properties extend into statistics, where you have the Gaussian distribution which feels both magical and universal, and is precisely the exponential of a (squared) Euclidean distance, i.e. exp(-(x - x0)^2).

I have the same feeling, that Cartesian coordinates and Euclidian distances are inherently connected as a natural pairing that is uniquely suited for producing the familiar reality that we inhabit and experience.

In my opinion it holds the same place in mathematics that water holds in biology and chemistry.

Cartesian coordinates and Euclidean distances are both great ideas, for different (orthogonal) reasons.

Cartesian coordinates are orthogonal, which is great.

Euclidean distance is great because it makes space flat and rotationally symmetric.

One of these ways (from which Cauchy-Schwarz and the other Hilbert Space results follow) is that d_2 is the only metric that satisfies the Parallelogram Law [1]:

2 d_2(x) + 2 d_2(y) = 2 d_2(x + y) + 2 d_2(x - y)

[1] https://en.wikipedia.org/wiki/Parallelogram_law

Remove the factors of 2 on the right, and square all the d_2.

Reply on news.ycombinator.com