A high-dimensional sphere spilling out of a high-dimensional cube
stanislavfort.github.io
stanislavfort.github.io
The spikiness of n-cubes is apparent when you look at the (solid) angle at each vertex; it starts constant (1/2 pi radian in 2D, 1/2 pi steradian in 3D) but reduces thereafter at an increasing rate (1/8 pi^2, 1/12 pi^2, 1/64 pi^3, etc.).
What's weird isn't the sphere, it's distance, and I think that's easier to process. Going from a (1d) sidewalk to a (2d) football field to a (3d) ocean, it's easier to see our intuitions about distances slowly breaking down.
How I am understanding this is that the non-uniformity is in the cube, and I think it's very helpful in visualizing it.
I can imagine in 3d, the centers of the sides of the cube being pulled in, so that it's kind of hyperbolic looking.
The diagram on the web page is just wrong. The enclosed sphere does not poke out with "spikes".
The error in the diagram is that it shows the corner cubes "filled" with spheres, but this is not what happens in the higher-dimensional analogues. If you take a cube-shaped cross section of a 4D example (through the center), you wouldn't see the corner spheres at all.
Think of a 2D slice through the middle of the 3D example: You'd just see a square, no circles!
Wait wait wait, I thought the "cube" had the same number of dimensions as the "spheres".
But cross-sections don't look the way you think they do. So if you draw a 3D intersection through the 4D case, it looks much like the 2D intersection of the 3D case, etc...
In all such "intersection" diagrams you don't see the spheres in the corners of the cubes. The conceptual diagram on that page showing the high-dimensional case shows the corner spheres and the middle sphere somehow "squished out" in a spiky way.
This is not at all what happens.
If instead you mean most of the mass of the ball is in the points which have all but one coordinate within epsilon of zero, then the intuition doesn't follow. It's equally true that you get most of the mass considering just points with all coordinates within epsilon. And for that you get that the mass of the ball is concentrated in an epsilon cube at the origin, which excludes exactly the spikes that you were basing the intuition on. The weird thing in this thought experiment is the epsilon cube not the sphere. For example the epsilon cube contains points much further than epsilon from the origin and so it's maybe not surprising that it contains most of the sphere.
that is kind of circular argument as "random" really depends on the density measure underlying the chosen sampling distribution.
I do agree that volume and probability distribution are essentially the same.
(That's also why at my university measure theory and probability theory were handled in the same course.)
https://www.math.ucdavis.edu/~strohmer/courses/180BigData/18...
> The result, when projected in two dimensions no longer appears convex, however all hypercubes are convex. This is part of the strangeness of higher dimensions - hypercubes are both convex and “pointy.”
Unstudied curiosity: steradians seem to be defined in terms of the point of a cone. But the corner of a cube is the intersection of several (for a cube, 3) planes, not a cone. How do you do the calculation of steradians?
To make a 2D analogy, you can think of 2D angles as a portion of a circle represented as a fraction of 2pi radians. Cut it in half and you have pi radians, cut that in half (so a quarter of the circle) and you have pi/2 radians, or 90 degrees.
That 90 degrees "quarter of a circle" example is looking at it as the "point of a pie slice", but it's the same 90 degree 2D angle as you have in the corner of a square.
You can look at the corner of a cube the same way. The full sphere of solid angle is 4pi, a hemisphere is 2pi. Now take that hemisphere and cut it into quarters (1/8ths of a sphere). Each of those quarters is pi/2 steradians, and the solid angle at the center is the same solid angle represented by 1/8th of the sphere is the same solid angle you have at the corner of a cube.
Or to put it another way, you could pack the corners of 8 cubes around a point and it would leave no empty gaps, so the corner of each cube is occupying 1/8th of a "full" 4pi steradians.
In the same way that an angle of pi radians corresponds to pi length around the perimeter of a unit circle, but if you're measuring on the perimeter of a larger circle you'll have a correspondingly larger length.
The "4pi steradians in a sphere" comes from the area of a unit sphere (4×pi×r^2 = 4×pi). I've never had cause to use 4+ dimensional solid angle for anything, but you'd determine full n-dimensional radians the same way, I guess with a 4-dimensional sphere the "surface area" is a 3D volume on the perimeter of the 4D unit sphere's space?
I understand this. It's not what I'm talking about.
There was a claim upthread that the corner of a square and the corner of a cube are equally sharp because (1) one of them is π/2 radians; (2) the other is π/2 steradians; and (3) π/2 is equal to π/2, hence equally sharp.
I'm saying I don't think this can be right, because when we consider two angles that we know are equally sharp, the 2D flat line and the 3D flat plane, one of them is π and the other is 2π, so unit-free radian measurements cannot be a valid measure of sharpness. The radian measurement needs to be normed against the full sphere in an equal number of dimensions, so that the flat line equals 1/2 (of a circle) and the flat plane equals 1/2 (of a sphere). I want to conclude that these two angles are equally sharp because 1/2 is equal to 1/2. But even if I can't do that, I need to avoid concluding that a flat line is twice as sharp as a flat plane, because I know that's untrue.
But my reasoning would tell us that the corner of a cube is twice as sharp as the corner of a square, 1/8 to 1/4.
You could make up some metric like "the unit sphere occupies a greater volume fraction of the unit cube compared to the area fraction of a unit circle in a unit square", and compare like that if you only have some fraction of each circle/sphere instead of the whole thing, but is there any value in making that comparison?
I think this is actually something we can't avoid. The problem with dodging the question this way is that it's really, really easy to translate a low-dimensional angle up into a higher-dimensional space.
Imagine I have two 2D vectors, U and V, with an angle between them. I can think of them as representing an infinite 1D surface consisting of the points (m·u_1 + n·v_1, m·u_2 + n·v_2) for all nonnegative real m and n.
It's trivial to define a 2D surface in 3D space representing exactly the same angle: it is the points (m·u_1 + n·v_1, m·u_2 + n·v_2, a) for all nonnegative real m and n and all real a. When the angle between U and V is θ radians, this surface will always express an angle of 2θ steradians. I don't think it's a stretch to say that the two angles, θ radians and 2θ steradians, must be equally sharp. (And indeed, the flat-line / flat-plane example is a special case of this one.)
https://news.ycombinator.com/item?id=12998899
https://news.ycombinator.com/item?id=3995615
Some of the comments here were made in those discussions, but some of the comments on those discussions have not yet been made here.
0. https://youtu.be/uU_Q2a0S0zI?t=1716
1. https://press.stripe.com/the-art-of-doing-science-and-engine...
Many of the missing comments from those linked submissions have also not been made here yet. Hope we'll see them soon.
Lines and areas are different animals; we cannot reason about them apples-to-apples; we know this intuitively. Areas and volumes are different animals too; we cannot reason about them apples-to-apples; we know this intuitively. Similarly, n-dimensional objects and (n+1)-dimensional objects are different animals; we cannot reason about them apples-to-apples.
As human beings, we find it so difficult to reason "visually" about higher dimensional spaces, in part, I believe, because our puny little brains have spent a lifetime learning to model three dimensions (with a fourth dimension, time, flowing only in one direction).
--
[a] See this comment by scatters: https://news.ycombinator.com/item?id=29969181
[b] See this old thread for intuitive explanations about how and why this happens with n-spheres as we increase the number of dimensions n: https://news.ycombinator.com/item?id=15676220
If you spend a year playing 4D Minecraft (I assume without googling that this exists), I suspect your intuition would be very much improved.
I've played with an animation of a wireframe hypercube that you could rotate around different axes. It was quite mind-boggling. But Gardner particularly raved about the mind-altering effect of viewing a rotating hypersphere. I've always wanted to view that, but I never heard anything about it again. It seems to me that a 2D projection of a hypersphere must look to all intents and purposes like a sphere.
Does anyone know where that article might be archived? Or where I can view an animation of a rotating hypersphere?
We are just used to project a cube in a way that prevents to see its linearly-spatial configuration (a projection messes up lengths), but if you preserve these lengths and "flatten" them instead, a cube will flatten out to a sort of a shuriken.
Yup. Even a regular 3D cube (and 2D square) has concave faces if you're viewing it from a polar perspective. As I stand in the center of the cube, measuring distances from me to the surface, I'll see that measurement follow a concave pattern.
Yeah, I know that's not the definition of concavity or whatever, but when relating a sphere to a cube, and trying to get an intuition of higher-dimensional spaces, I think it helps to look at it from the sphere's perspective rather than a Cartesian one.
It seems reasonable to extend that same intuition to n-D sharpness/spikiness in an accurate way. Adding an extra “face” just chops more off making those vertices sharper and sharper, at least relative to the high dimensional space around it.
However, it's clear the "starfish" intuition is simply not accurate. That's not what N-cubes look like. The point of this post is that we should have cognitive dissonance when we try to think about 10-cubes, because it's weird that (A) the "inner sphere" pokes out of a shape that is (B) convex everywhere. You can resolve the cognitive dissonance easily by simply ignoring or rejecting B -- sure, it's not weird that such a sphere would poke out of a starfish. But you are wrong. It's not a starfish! It's convex everywhere! So you can't say "why do y'all have cognitive dissonance about this?"
That's probably a stupid question, but while that fact is intuitively obvious for D={2,3} -- as this problem tries to demonstrate -- higher dimensions are unintuitively WEIRD.
And all the other spheres are simple symmetrical mirrors, so how could it not touch all of them at the same time it touches one? That should scale to an arbitrary number of dimensions, right?
There are 2^k > 512 spheres stuck to eachother across k-d (pretend k=9). The line from the center to the point where the inner sphere touches one of outer spheres has to shortcut through all k dimensions to get from the center to the sphere.
This distance has been massively inflated due to the number of dimensions. But the distance to the edge of the box hasn't been inflated - it's just constant, so the inner sphere breaks out.
I'm not trying to do research here, I'm just boggling at the unintuitive result, and trying to see if there might be a flaw in the chain of logic. The fact that this is "well known" is enough to scare me off from barking up this particular tree.
To put it another way, it's not obvious to me that the point of contact between the inner pink circle and the outer black circle is along the green line.
Then the result follows because all the spheres are defined as centered on the cube/sub-cubes respectively.
That said, there is a symmetry argument that if it were centered anywhere else, something is wrong. But that only works if there is only one unique sphere that touches all the other spheres, which is also not obvious to me in higher dimensions.
Hopefully this was the correct amount of words and equations that you can reconstruct this on paper. I think doing it this way shows you that the outer spheres get increasingly smaller relative to their containing "quadrant". For me, it also elucidates that the final drawing in the post is very misleading because the hyperspheres don't spike like that. Rather, they look more like 4-leaf clovers and that is what allows it to escape the cube in the additional "horizontal space" of the aforementioned quadrant.
I've got a couple, maybe (depends on your intuition i guess):
In 4d a (topological) sphere can be knotted.
Hyugens's principle: When a wave is created in a field in N-dimensional space, if N is even, it will disturb an ever-expanding region forever (think a pebble hitting a pond's surface) whereas if N is odd the wavefront will propagate forever but leave no disturbances in its wake (think of a flashbulb, or of someone shouting in an infinite space full of air but no solids to echo off of).
Sometimes called "the flaw of averages". Of course I learned about this from another HN post recently:
https://www.thestar.com/news/insight/2016/01/16/when-us-air-...
Arguably you can do that in 3D, if you accept the horned sphere as a knot. Though I suppose that does raise the question of what you are willing to call a sphere.
Regardless it's an embedding of a sphere that cannot be deformed into a unit sphere so I think the analogy holds.
For example the volume (hypervolume) gets concentrated close to the surface of the sphere when dimension grows. For example, if you have symmetric multidimensional probability distributions around the zero it becomes weird.
When n increases the mass of the probabilities are around a sphere of radius n^0.5, nowhere near the origin.
I love this channel :)
This is an example of a misleading first example. What actually will happen and what you expect will happen are not the same, because you got tricked by the first example.
I mean, everyone knows that everything is super far apart in higher dimensions, it surprises me that it takes up to 10D before he slips out-- perhaps more intuitive when i keep in mind that n-spheres are the most compact shape?
Am I missing something? It to me that only r*D grows as we increase the dimension, not r by itself. Since we don't seem to get r > 2a for any D, I don't really get the conclusion that it's sticking out of the cube
I don't know when that happens with the simplices. I assume the middle sphere pokes thru before it does with the hypercubes, since the simplices are pointier.
If I've done my calculations right, putting n+1 n-spheres in the corners of an n-simplex with unit sides so that they're tangent to one another gives them a radius of 1 / (sqrt(2n(n+1)) + 2), and then if you put a sphere in the middle tangent to all those it has radius [sqrt(2n/(n+1)) - 1] times this.
So for very large n, the "corner spheres" have radius of order 1/n, and the "centre sphere" has radius about sqrt(2)-1 times the radius of the "corner spheres", and both of these -> 0 as n -> oo.
(But! "If I've done my calculations right" is an important condition there. I make a lot of mistakes. If you actually care about the answer then you should check it.)
None the less, it's a fun thought excercise! Thanks
The Mahalanobis distance is a measure of the distance between a point P and a distribution D, introduced by P. C. Mahalanobis in 1936. It is a multi-dimensional generalization of the idea of measuring how many standard deviations away P is from the mean of D. This distance is zero for P at the mean of D and grows as P moves away from the mean along each principal component axis. If each of these axes is re-scaled to have unit variance, then the Mahalanobis distance corresponds to standard Euclidean distance in the transformed space. The Mahalanobis distance is thus unitless, scale-invariant, and takes into account the correlations of the data set.
Euclidian distance measures the distance between two points, while Mahalanobis measures the distance between a distribution (canonically multivariate normal) and a point. Mahalanobis distance is not a generalization if Euclidian distance, it's an altogether different concept of distance that doesn't even make sense without talking about a distribution with mean and covariance matrix.
> Euclidean distance measures the distance between two points, while Mahalanobis measures the distance between a distribution (canonically multivariate normal) and a point
In a discussion about metric and metric spaces we dont care about those things, its abstracted out and considered irrelevant. All that matters is that we have a set of 'things' and a distance between pairs of such things that satisfies the properties of being a distance (more precisely, properties of being a metric).
@CrazyStat (I cannot respond to your comment so leaving it here)
I think you overlooked
> things that satisfies the properties of being a distance (more precisely, properties of being a metric).
that I wrote. Of course it has to satisfy the properties of being a metric. The red herring, as far as dimensionality is concerned, is the complaint that Mahalanobis is defined over distributions while Euclidean is over points.
The part about MD that you get absolutely right is its nothing but Euclidean distance in a space that has been transformed by a linear transformation. MD (the version with sqrt applied) and ED aren't that different, especially so in the context of dimensionality
@CrazyStat response to second comment.
It indeed isnt, its just Euclidean distance under linear transformation. I was just quoting you, you had said
> while Mahalanobis measures the distance between a distribution
My point was even it is defined for distributions its not really relevant.
> Mahalanobis "distance" is more closely related to a likelihood function than to a true distance function.
That's a subjective claim, and open to personal interpretation. Mathematically MD is indeed a metric (equivalently a distance) and it does show up in the log likelihood function. Mahalanobis was a statistician, but MD is a bonafide distance in any finite dimensional linear space, with possible extensions to infinite dimensional spaces by way of a positive definite kernel function (or equivalently, the covariance function of a Gaussian process)
This is not a red herring, it's a fundamental issue.
Mahalanobis "distance" is more closely related to a likelihood function than to a true distance function.
Like Mahalanobis distance, apple pie is not a fruit and is not a generalization of an apple.
In short, much that I love Mahalanobis distances' many properties it does zilch for dimensionality.
https://stats.stackexchange.com/questions/99171/why-is-eucli...
What do you mean by "Needed" vectors? Distances are generally scalars, no?
Can you give an example of how we've made use of the 4th, 5th, 6th dimension, etc.?
[0] https://en.m.wikipedia.org/wiki/Kaluza%E2%80%93Klein_theory
edit: I agree, I don't know why you got down voted. It was a thought provoking question.