Divide by depth for instant 3D | Hacker News Reader

Divide by depth for instant 3D

gabrieloc.com

218 pointsgabrieloc39 comments

The explanation of "What's that extra 1 for?" in the column representation of 3-d coordinates (x y z 1) could benefit from mentioning that translation—moving things—is not a linear transformation (the origin is not mapped to itself) but an affine transformation. Therefore, you cannot represent translation in 3-d space with a 3x3 matrix. What you can do, though, is embed that 3-d space within a 4-d space fixed at some coordinate on its 4th dimension, typically w=1. Then, a translation in the original 3-d space can be represented as a linear transformation in the 4-d space and thus can also be represented by a 4x4 matrix multiplication. So the extra 1 is actually what allows all common 3-d operations, including translation, to be done via linear algebra and thereby harness the brutal power of matrix multiplication on modern computing devices.

You can do rotations and translations "via linear algebra" without homogeneous coordinates (the 4-element tuple representing a 3d point or vector), since adding two vectors is linear algebra.

What this lets you do is composing multiple transforms into one matrix multiplication instead of a sequence of multiplications and additions; that's what dramatically increases performance, on modern computing devices but most especially on ancient ones, where we were fighting for every MUL.

More details: https://gabrielgambetta.com/computer-graphics-from-scratch/1...

You don't need to use homogeneous coordinates to represent a series of linear transformations as a single matrix. That's just basic linear algebra: any series of linear transformations is itself a linear transformation, and any linear transformation can be represented as an equivalent transformation matrix.

But you do need homogenous coordinates if you want to include translations (fixed distance shifts, such as moving the camera) in the set of operations you can represent as linear transformations and thereby gain all the benefits of linear algebra, including the benefit of being able to include them in a series of operations that you can represent with a single transformation matrix.

Rick Szeliaki’s “Computer vision algorithms and applications” has a lot of great motivation for homogeneous coordinates in the first chapter and appendix. For example, in 2D, the cross product between two points in homogeneous coordinates is the line joining them, and the cross product between two lines in homogeneous coordinates is their point of intersection.

its linear algebra but not a linear transformation. repeat: translations are not linear transformations in 3d.

nice intuition there, this comment prompted me to consider a simpler example, 2d embedded within 3d. does the 2d plane (embedded in 3d) go through the 3d origin (where 0 maps to 0) and is thus a linear transformation in 3d but a 2d affine transform in 2d? it feels like this is the case?

No, the 2d x-y plane in your example cannot pass through the 3d space’s origin because that would imply that you fixed the z coordinate at zero. The plane must be fixed at some nonzero z because you need to be able move x and y values by some scaled version of z to make translation happen. If z is zero, that scheme does not work.

Consider a transformation f where we wish to move x-y coordinates s units to the right. In 2d, we could express it as:

f(x, y) = (x + s, y)

But that transformation is affine not linear. There is no way to generate the value s as a linear combination of the inputs x and y. So, our workaround is to embed the x-y plane into 3d space at z=1. Then we can move (x,y,1) points in that plane s units to the right using this transformation:

f(x, y, z) = (x + s*z, y, z)

This new transformation is linear: it maps (0,0,0) to itself. But it maps our embedded 2d plane's origin (0,0,1) to (s,0,1), shifting it right by s units, as we want.

The matrix form of that transformation is:

    [[1 0 s]
     [0 1 0]
     [0 0 1]]

The same scheme would work if we had embedded the plane at any fixed z=r for nonzero r. We would only have to rescale the s in the matrix to s/r. Again, however, if r=0, this scheme will not work, as 1/r has gone to infinity.

Yes, affine transformation matrices are essentially shears.[0] In the 2D case, shearing the plane z=1 in 3D space essentially translates it around.

[0]: Here’s a visual: https://gunn-gatm.github.io/textbook/gatm.pdf#page=28

ive been thinking -- if i have a 3x3 matrix [1 0 q; 0 1 r; 0 0 1], when dotted with x, those rows are planes in 3d with normals n1=(1 0 q), n2=(0 1 r) and n3=(0 0 1). plane 3 is parallel to the x-y plane and 1 unit up. plane 1 is tilted by q and parallel to the y-axis, plane 2 is tilted by r and parallel to the x-axis. the intersection of those planes (i.e. the solution x) when calculating Ax=b gives an output vector b sitting in 3d space at b=(x+qz, y+rz, 1*z). since we always specify z=1 we have b=(x+q, y+r, 1). in 3d this is a linear shear because we are translating proportionally by z but because z always equals 1 in this case we effectively get a translation in 2d.

so as the other 2 helpful commenters also just said: 3d shears using linear algebra degenerate to 2d affine transformations when z=1 (or w in 4d)

re-read this a day later and ofc it still makes sense to me but really most of it is barking up the wrong tree, focusing on solutions (which we already know) rather than outputs. if i could delete it i would but oh well.

Is that why quaternions are interesting for computer graphics? I studied mathematics and computer science but never understood (or even looked so hard into) the connection.

Kind of! The scalar allows vectors and scalars to be expressed together as one object, which has a lot of computational and mathematical niceties.

The killer feature is that you can put the two together for a rotation axis (~ vector) and angle (~ scalar).

With just three gimbals (rotating circles), if gimbals A and B are aligned, you only have two degrees of freedom (rotating A is the same as rotating B). Because of this, interpolating angles is unwieldy in vector space. 'Gimbal lock' confounds animation (in hilarious but unrealistic ways) but also aerospace (four hours before 'one small step for man', just after landing, Collins joked he would like a fourth gimbal for Christmas).

FWIW, if you start with "The view frustum is a 90 degree pyramid with the tip cut off at z = 1 and the 'end' at infinity", you can then work out how to map that to a NDC using a matrix and perspective divide.

I've used that when teaching short "Graphics 101" (not in the first session though) and the math comes out more intuitive than the usual "here's how to calculate a perspective matrix, don't ask where these numbers come from" version.

I tend to do it with a window.

Students can easily imagine the pyramid from an eye to the window frame, and that it keeps going.

Then I point to an object outside the window and say imagine strings going from the corners of the object to your eye. To do this they would have to go through the window, where do they do that. 3d graphics is finding out where on the window the strings go through so you can stick a picture of thing outside onto the window and it looks exactly the same.

For me learning on my own, it was even illustrative to not use a near plane and see how things behind the camera would still appear in front of it.

A lot of 3D graphics can be derived pretty easily just from knowing a few basics like divide by depth. I think knowing how to construct a transformation matrix from a coordinate system basis is another one -- that would remove the need to look up how to construct a perspective matrix, for example.

A few things like that will get you pretty far and you can kind of take the same journey of discovery as early 3D pioneers. That's one of the best ways to learn because you're much more likely to remember something you figured out compared to something you just read about.

It gets tricky with perspective-correct textures, but running into issues like that on your own (even if not solved on your own) is part of the fun of learning, I think.

Trust me, I am not very bright. But as boy with a 486 in the mid 90s, I was so inspired, a that I figured this out without the internet (was still unheard of) or the books.

I still remember the rush that I got when I got the rotating cube in my QBasic IDE in my 14inch monochrome monitor. The intuition I used was derived from how light casted shadows of 3d objects in a 2d wall..From there, this x' = x/z y' = y/z, follows naturally..

fabulous!

I read your other comment below where you describe similar experience..Fun times, right? I am sad that kids today will never know that..They are born into a world where computers are not seen as programmable devices..and just as a glorified TV.

Hi! I wrote a few notes on how 3D cameras work with interactive examples to hopefully demystify a pretty complex topic that I once struggled with. Maybe this is useful for someone here, and if not, there are fun sliders to play with!

The visuals and sliders are great!

Maybe consider clipping the ball properly on the edges of the sides of the view frustum ?

Similarly the near and far could also be clipped; i know this is not true of the math necessarily but is the expected result in 3d graphics applications.

Really nice explanation. When I saw the title I immediately remembered the tsoding video as well so it was nice to see you mentioning it

At one point all of this seemed like common knowledge in software because Carmack, Abrash and Hecker (among many others) were working in the open on games and discovery. Kind of funny that someone had to reinvent from first principles!

The issue is it’s hard to find a good simple explanation. If you search you’ll more often then not get either a bad explanation, one that doesn’t work for you personally but might for others, one that’s too mathy and assumed too much prior knowledge, etc…

Now-a-days maybe you can ask an LLM for something exactly at your level and anything you don’t get you can ask it that too. That wasn’t the case until recently and many still don’t use them

” that’s too mathy” I’m not really sure there is any remedy to the fact computer graphics is based on math.

I suck at math myself so can relate. But actually the math you need to learn to be effective at graphics is mostly about understanding _a very small subset_ of a bunch of concepts (such as how 2x2, 3x3 and 4x4 matrices work) and one is set for life,

so I think it’s good to give the formal mathemathical explanation and then Gilbert Strang what it all means.

One just needs to pick any book on computer graphics written in the past 40 years or so.

But camera transform is so unintuitive it really helps if the student tries to work it out for themselves why it works.

For me it’s really weird anyone would write code and _not_ start from reading all of introductory computer graphics literature because it’s so awesome but I realize I’m the outlier here.

A fun demo of this from earlier this year (Tsoding): https://www.youtube.com/watch?v=qjWkNZ0SXfo The good stuff starts around 7 mins, but it’s a great presentation and it’s almost magical how everything comes together.

(I somehow missed that OP cited this at the start, but it’s a great companion piece)

Like the post, I wrote an old-school 3D engine that does the same math—renders flat-shaded polygons in an HTML5 Canvas.

Demo: https://engineersneedart.com/Phosphor3DTest/

(cursor keys drive the sand crawler, square-brackets change FOV)

Sources: https://github.com/EngineersNeedArt/Phosphor3D

(I was seeing a few anomalies and sent Claude to investigate—found a math error or two. There are still some anomalies in depth sorting the polygons but not due to the code, I believe—instead the model itself.)

It reminds me my "3D engine in 47 lines ov javascript", which can only draw line segments :) https://jsfiddle.net/06L845jx/86/

And 54 lines of javascript: https://jsfiddle.net/06L845jx/127/

Original discussion: https://www.reddit.com/r/javascript/comments/9nihtw/a_simple...

OMG This how I got 3D wireframes working on my zx spectrum in 1989! Linear Algebra - what's that? I got it working by pure experimentation - no coding high has ever quite reached the same level since...

This is a beautiful post, good work!

Really loved the examples of the math being put into action with those lovely little sliders

Awesome. To take this a step further, I used this math to turn 2D photos into 3D scenes with depth maps: https://blog.skz.dev/3d-reconstruction-from-public-photos

3D on 2D screen is all about giving brain enough depth cues. Having size diminish with depth is a big one. The 2nd biggest thing is occlusion, which is altogether skipped in these wireframe renderings. Another big one is fog, or impact of medium the light is traveling through, very easy to do for the basic effect. It's basically just fading the colors to gray when more distant from camera.

You can have a convincing 3D with just the occlusion and fog (or even just the fog), without simulating the perspective. It's just another artistic direction.

Shadows can also be pretty important for something feeling like a real 3D scene.

Without them even pictures of the real world can look flat: https://en.wikipedia.org/wiki/File:Lahaina_Noon.jpg

Good point. And ambient occlusion, as sort of many micro-shadows. A bit more advanced to implement.

The sliders are great UX on mobile, love the detail to attention.

I remember doing this in generative computer graphics class, i thought it was so neat, just dividing by the w component to get correcti projection.

Great writeup.

This is very cool.

Now I want to write a game that does all 3d projections using x' and y' only.

wtf, I thought this was THE cheap way to 3D.

Reply on news.ycombinator.com