Path Tracing vs. Ray Tracing

So, basic Whitted style ray tracer is, for each pixel, project a ray into the scene and see what the closest intersection is. Now shoot secondary rays at each light source to see if the intersection is in shadow for that light, sum the lighting accordingly. Also if the material hit is shiny, shoot a ray out using snells law and recurse to see what it sees.

Now, with path tracing instead we approach the rendering problem more generally, and think of it as a 6D integral that describes all possible ways light can bounce in the scene. Then we sample this integral using monte carlo techniques. The most naive version of this would be to emit virtual photons from lights, trace where they go probabilistically using scattering equations at each surface interaction (so not just snells law) and then seeing if it ever blunders into the virtual camera.

Obviously that's very inefficient, so practical path tracers work backwards tracing paths from the camera, or bidirectionally doing both and seeing if they can connect. Then there's a huge pile of techniques like importance sampling that help make all this more efficient.

That said, at the end of the day, path tracing is much more computationally intensive than the classic ray tracer. However it's worth this because it can truly render all lighting in a physically correct way. Classic ray tracing bakes in assumptions about shadow rays and simple reflections, which is why images made with it have a very stark unrealistic appearance (think sphere and pyramid floating over a checkerboard style images).

Let's start with a ray. It begins at the camera, travels through some point in the viewport, and goes on its merry way through the scene. It does so in a straight line, and Interesting Stuff happens when the ray hits some object. In practice by far the hardest part is figuring out what you are going to hit - and doing so quickly. But we're going to completely ignore that here for simplicity.

The most primitive way of ray tracing is to draw a pixel colored by the object you hit. The ray hit a red ball? Pixel is red. The ray hit a green cube? Pixel is green. The ray hit nothing? Pixel is black. But this creates solidly colored objects, so it is pretty useless. One important thing to note is that you are done after a single ray.

Let's add some basic physics to it. Perfect mirrors are easy: the angle of incidence is equal to the angle of reflection, so we just alter the direction of the ray and continue tracing it. Transparent objects are easy too: Snell's Law will trivially tell you how the ray should be adjusted to account for different refractice indices - sometimes it gets bent, and sometimes it gets reflected. And we just got things like lenses implemented! Adjusting for things like colored mirrors or tinted water is left to the reader. Opaque objects are slightly more complicated. They get their light from, well, light sources. To do this, we cast secondary "shadow rays": one ray from each point light source to the point on the opaque object we hit. If nothing is in the way we use a shading algorithm like a Phong shader to determine the influence of that light source at that angle on that object. The important takeaway here is that the ray tracing is now recursive. We have now arrived at what we call "Whitted-style ray tracing".

There are two important things to keep in mind for Whitted-style ray tracing. First, you have to shoot a shadow ray to every light source. This gets expensive really fast. Second, while you can create objects which do multiple things at once (somewhat opaque, somewhat glassy), this results in even more rays. In practice, hitting a surface point spawns 1) a shadow ray for each light source 2) a reflection ray 3) a ray transmitted through the material. And the reflected and transmitted rays will go on to hit even more surfaces - leading to an exponential growth in the number of rays. And the final result isn't even physics-accurate!

Finally, path tracing. We start off the same, but when we hit an object we choose a single random path to continue on, until we finally hit a light source to terminate on. The key point here is that the continuation vector is a weighted random value of a probability function: if enough rays hit this object, the average result will eventually approximate the integral of this function. Similarly, if an object does multiple things, like both reflection and transmission, we randomly choose one of them to continue with. This means we can simulate complicated physics without generating an exponential amount of secondary rays. The only downside is that you need to trace a lot of paths to get a good result: a single path can end up with a heavily biased result due to the randomness rolling dice in a silly way, which ends up generating noise in the output.

----------------------

In summary, ray tracing recursively generates secondary rays to exactly calculate some laws of physics, leading to an explosion of the amount of secondary rays you need to handle, and poor results because you simply cannot implement all physics correctly. Path tracing generates a predetermined and fixed amount of rays randomly going through the scene in an effort to eventually approximate pretty much all laws of physics, but you need a lot more primary rays to get a good approximation and thus a noise-less result.

I followed the Advanced Graphics course at Utrecht University a few years ago, taught by the amazing Jacco Bikker. It seems last year's recordings are available (in English!), so if you want to know more check them out at http://www.cs.uu.nl/docs/vakken/magr/2021-2022/index.html

At each intersection, a ray tracer might spawn a refraction ray, a reflection ray, and a shadow ray towards each of the lights. The reflection and refraction rays might spawn rays of their own, recursively. There are pathological cases where this gets out of hand, so most ray tracer have a maximum recursion depth or some threshold that if the ray doesn't contribute more than some threshold to the color of the pixel it just defaults to black (or whatever) and isn't computed.

Most scenes aren't made entirely of shiny glass spheres, so this usually isn't a major problem.

Path tracing is a little different, instead of evaluating a recursive tree of rays, you just do a lot of random sampling. Instead of evaluating both the reflection and refration ray, you flip a coin and evaluate one or the other. Or if the surface isn't perfectly smooth and glossy, you pick a random direction. Generally, path tracers have to trace a lot of rays per pixel to get anything other than a very noisy image. That can take a long time, so historically path tracing has been a lot slower than classical Whitted-style recursive ray tracing.

In either case, path tracing or ray tracing, the traversal doesn't (usually) terminate in the ray colliding with a light source. Instead, at some hit location you do a direct illumination test which usually amounts to casting a shadow ray at all the light sources to see if that spot is directly illuminated, and then evaluating the texture and lighting at that point to produce a color, which propagates back up to the final result.

> I don't understand this either. This just sounds like raytracing without reflections.

Yep, that's raytracing without reflections, refractions, or shadows. Or basically anything that distinguishes it from plain polygon rasterization. Which is why ray tracers and path tracers do actually compute those secondary rays, they just have different strategies about which ones they evaluate.

Plain Whitted ray tracing handles some additional optical effects that polygon rasterizers aren't good at (reflection, refraction, shadows), but it's not a complete solution. (Or we would say that it's not a full solution to the "rendering equation", because it omits some important terms.)

Path tracing on the other hand is a complete, unbiased solution to the rendering equation. (Which is to say it renders a "correct image" according to a fairly realistic model of how light behaves, given infinite compute power, and if one's resources aren't infinite, at least it will give a statistically representative if perhaps noisy result.) The main thing path tracing handles that ray tracing doesn't is global illumination: basically, light that bounces off more than one surface between the light source and the camera. It can even correctly model semi-glossy surfaces. Global illumination can often make the difference between "this looks CG" and "this look like a photograph" (assuming there aren't any other major tells).

https://en.wikipedia.org/wiki/Rendering_equation

"Raytracing" is a bit of an overloaded term, but conventionally "Raytracing" for graphics meant a serious of "raycasting" events, and "Whitted" style was a technique which could approximate reflective and refractive materials fairly accurately (as long as you modelled the fresnel effect and absorption) in a recursive style. Modifications on top of this are also to do limited indirect rays for global illumination and this was used a bit (in film/CG VFX in hybrid renderers like PRMan pre 19), but doing global illumination this way was pretty slow and didn't scale well (an exponential branching factor at each bounce). So technically, the article is sort of correct on this point, as Pathtracing is more smart about how it handles the full rendering equation of "all" light transport simulation, and is faster than Raytracing for high fidelity (i.e. global illumination, area lights), due to its stochastic nature of sampling multiple dimensions of lights, surfaces, volumes, etc across many samples per pixel.

Pathtracing does take reflection and refraction into account, but generally (there's a technique called 'splitting', which sort of branches the paths to make it more like 'Whitted' raytracing, but ignore that for now) it relies on "importance sampling" to make smarter decisions, based off the material response (diffuse, glossy, reflective, refractive), which guides the direction of the next ray off the previous surface. This generally means that due to the "random" nature of pathtracing sampling (it uses monte carlo random sampling), predominant lighting is generally more efficient, although things like caustics, SSS and indirect illumination can still be very noisy as they're hard to sample and can take thousands of samples to resolve to "clean".

There are other techniques like Russian Roulette to kill particular paths which don't have much throughput and/or light contributions, which can speed up renders as well, and I think is what the "dumping most or all of the secondary ones doesn't affect things as much as one might think." part is referring to.

This session has one of the best overviews of Lumen. The previous year has a really good overview of Nanite.

In traditional rasterized games, the only thing that matters is whether the end result looks good. This means adding things like global illumination (a magical diffuse light source which lights up stuff regardless of position), or simply making some parts of a texture brighter to make it look as if a light source is shining on it. Things like shadows are mostly pre-baked into the textures, and primitive reflections can be faked by just mirroring whatever is already on the screen. Basically, fake it until you make it.

With path tracing, the renderer follows the laws of physics. If something is lit up, the light actually has to come from somewhere. And as it turns out, most light will have hit multiple objects before finally ending up at the viewer. The artist now has to make a reasonably physics model for every single object in the game - even those the player might not directly look at. It's a completely different approach - made even worse because we aren't yet at a point where every gamer has a computer which can do 100% path tracing, so you still need to support traditional rasterization for people without bleeding-edge GPUs.

To get an example of the visuals possible, look at modern cinema. Disney and Pixar movies are path traced, as is probably all CGI these days. I expect games to get a more diverse look over time, as artists get more experience with it.