Hacker News Reader

Comment by chrisseaton | Hacker News Reader

Can anyone give me any intuitive understanding of how ray tracing can be adapted into a conventional rasteriser workflow?

Doesn't ray tracing need a persistent, whole-world scene graph? Rays can end up anywhere, no matter where the camera is pointing, isn't that right? I thought the whole scene wasn't on the GPU? I thought the CPU streamed polygons to be drawn based on what was likely to be visible, but it can't stream the entire world every draw can it?

View on HN

The data for each scene hasn't been streamed to the GPU in a long time, not since the first few generations of GeForce cards. "Vertex buffers" store the geometry in the GPU's RAM, so it's there. The game engine just swaps out buffers as things come in and out of view.

I have the same questions as you regarding how this works with respect to offloading the the scene graph data structures though.

Looks like the GPU stores collections of triangles then used to perform the ray computations.

https://devblogs.nvidia.com/practical-real-time-ray-tracing-...

To shoot rays in scene you create acceleration structure (internal to driver/GPU hardware). Input to this structure are vertex & index buffers & model transform matrix. Geometry is specified as triangles or AABB. You can update this structure later by providing new transform matrix. There are different flags that control performance/memory usage - fast tracing or updates vs less memory usage.

See following links:

* https://www.khronos.org/registry/vulkan/specs/1.2-extensions...

But efficient ray casting requires a scene graph structure--an octree, or a bsd-tree, etc. Are the details of that just handled by the driver?

Yes. Driver/GPU handles that. It "just works". You just manage things on high level - what can be moved in world, what cannot and group the updates together.

> Rays can end up anywhere, no matter where the camera is pointing, isn't that right?

Only indirect or secondary rays. Primary rays (the ones from the camera) can only hit things that are visible to the camera.

Also, ray-tracing trivially allows for breaking up the rendering in multiple steps.

You could for example have a pre-made acceleration structure for the static part of the world, a less efficient but fast-building one for dynamic objects in the world, cast against either and see which intersection has the shortest distance. This could be done in parallel, and you can be more clever about it too depending on circumstances.

Not saying that's what people do, I've been out of the game for too long, but at least it's an option.

> can only hit things that are visible to the camera

Isn't the whole point of ray tracing things like refraction? Otherwise why not just rasterise?

I think you're generally correct, but that games studios and animators don't care much about refraction specifically (although it does make for cool tech demos). They care more about reflection and lighting in general (spotlights such as windows, true area lights - no point lights disguised as area lights using emmisive textures)

Ray tracing can be used for different camera projections, sharp reflections, sharp refractions, sharp shadows and other things like physics. (Also it's rasterize)

“...rasterize” — that’s American, rasterise is correct in UK and countries that follow their English spelling.

> Ray tracing can be used for different camera projections, sharp reflections, sharp refractions, sharp shadows and other things like physics.

I know. But you may need geometry that isn’t trivially visible in order to render these things.

> (Also it's rasterize)

I’m British. Not everyone does things the American way. Don’t try to push your approach on everyone else.

People keep giving you the answers and you don't seem to accept them. You said you thought it was just useful for refraction, this is what it is useful for in games, including different camera projections.

Another person replied and told you that the acceleration structure can be dynamic with geometry inserted and taken out.

> You said you thought it was just useful for refraction

"things like refraction". Not "just".

You said you thought ray tracing was never used for direct rays and only for indirect days along with asking how the rays have access to the full scene geometry and both questions have been answered thoroughly.

> You said you thought ray tracing was never used for direct rays

I didn't say this anywhere. You're imagining extra words between what I actually said. Everyone else in this thread understood my question, and they managed to answer it without also ignorantly trying to correct my spelling.

You were told that rays from the camera would only hit what is visible to the camera.

Also you might want to get upset at Nvidia, AMD, Pixar and every book on computer graphics if getting offended over spelling what you want to focus on.

(Also spelling is not an approach to something)

> You were told that rays from the camera would only hit what is visible to the camera.

But what is visible to the camera from rasterisation is different to what is visible to a camera using ray tracing! That was the point!

> you might want to get upset at Nvidia, AMD, Pixar and every book on computer graphics if getting offended over spelling what you want to focus on

You were the one who first got offended by spelling and tried to correct me to do it your way!

>>But what is visible to the camera from rasterisation is different to what is visible to a camera using ray tracing! That was the point!

That is not true.

>>You were the one who first got offended by spelling and tried to correct me to do it your way!

I wasn't offended, I just told you the way that it's spelled by the vast majority of people who invented and work with these things. It isn't 'my' way, I'm not sure why you are so hung up on this. Maybe you are trying to avoid answers you don't like to the incorrect assumptions in your questions.

> That is not true.

No it is true - a polyglon not visible in rasterisation could become visible in ray tracing if the ray passes through a refractive material. So you may need additional geometry. Get it?

Those are secondary rays, which are separate from the primary rays from a camera. Secondary rays from reflective or transmissive effects are not considered visibility from the camera. This distinction was very important in hybrid renderers that combined rasterization from the camera with ray tracing for secondary rays, like PRman and mental ray. Now we have the same patterns being repeated in real time rendering. This was explained to you already. You could learn something if you didn't think you knew everything already.

One of the turning points was to actually work backwards: instead of calculating every ray from a light source, then throwing away anything that doesn’t hit the camera, ray tracing now starts with the ray endpoints that end at the camera, thus saving all that work that needs to be thrown out the other way. Reflections do need access to the geometry off-camera, but it’s accessible in some for or another by needing to be available when the player looks at it, as well as needing to exist for gameplay interaction (such as an enemy shooting from off-camera)

Actually once you get a sufficient number of triangles, it can be faster to ray-trace than rasterize, as rasterized triangles have to be drawn, while ray-traced triangles can be skipped thanks to the acceleration structure.

Anyway, yeah sure the fun is typically doing secondary rays, but you could do those with the tricks I mentioned, and/or with reduced detail (thus faster to update) unless you're doing perfect reflection/refraction.

On top of what other people wrote, I think a common misconception is that raytracing is only useful to determine the final shaded color of a pixel on the screen. In its current form, things like RTX are still way too slow to do this at any appreciable quality level.

You can use raytracing passes at very different resolutions, quality settings, geometry detail levels etc though, and combine them with the 'regular' render pipeline later. For example to calculate environment maps for reflections, shadow maps, or even things completely unrelated to drawing (hit tests, sound reflection etc). Suppose I want to have a nice reflection in a pool of water for example, you can already get a very good looking effect by raytracing a low-resolution environment map from the water pools' perspective, which wouldn't need any tracing of secondary rays etc, and use it in your water shader.

There's uncountable ways something like this could be useful (not all of which can be accelerated very well though, I guess)

For doing the more traditional raytracing at any resolution you will always need an acceleration structure like a BVH though, but building/updating that is part of these kinds of hardware extensions.

Reply on news.ycombinator.com