How much boilerplate code you need to write a ray-tracer?
sergeyreznik.github.io
sergeyreznik.github.io
I think that's the most impressive little demo I've seen in ... quite some time. I can think of more impressive ones, but they're all extraordinarily complicated in comparison.
How'd you do the glass if there are no meshes? I'm about to pass out, else I'd dig in.
I'm especially curious how you're getting the refraction right. I forget my basic physics, but is it as simple as deflecting the ray slightly? But then how does it work on a curved surface? I guess it happens per pixel, so it shouldn't be too surprising, but... still, I was expecting a lot more aliasing than this: https://i.imgur.com/fdDVcCT.jpg
I guess I've never seen a real time raytracer that happened to trace through ripply glass, which is why it feels so neat.
What's much harder to do is simulating the light that's refracted or reflected off of objects (called "caustics"), like the light on the bottom of a swimming pool. To do that in a physically correct way generally requires falling back on some kind of global illumination technique like path tracing or photon mapping.
Think of a simplification in 2D. Sunlight comes from directly vertical above a sine wave. For every x-coordinate where you cast a sun ray, you just need the slope of the sine-wave at that x-coordinate to calculate the angle for what will happen at the intersection.
A mesh isn't any fundamental unit of physics or geometry, it's just our customary way of approaching the rendering to get good-enough fast-enough results from our current hardware.
For a very simple form, you can sample a specific wavelength with each ray, trace it using wavelength-dependent refraction indices (e.g., from the Sellmeier equation) whenever you compute refraction or internal reflection. Then tint the ray by the color and visible intensity of the wavelength when averaging it into the image pixel.
For a lot more detail, see e.g., "Efficient Spectral Rendering on the GPU for Predictive Rendering" (https://hal.inria.fr/hal-03331619/document).
I find this code base extremely easy to read despite not being a C++ (It has that Carmackian quality). Pretty easy to use to port to the language of your choice. https://github.com/TomCrypto/Lambda To get the kind of effect you see in the youtube video you'd render diamond or something with extreme refraction.
This is far from real time though of course. It will run an order of magnitude slower than anything you can do with full spectrum rays. So doing it in realtime would probably be just like subsurface scattering is done in real time: you just have to cheat. Perhaps there can be some inspiration from the real thing though.
[0] https://fabiensanglard.net/rayTracing_back_of_business_card/
I never wrote ray tracers, but in my experience the best way to parallelize things is usually “none of the above”.
C++ runtime has a built-in scheduler, OpenMP https://www.openmp.org/ If the algorithm needs to dynamically spawn tasks (graph search, flood fill or similar), Windows kernel has another one, see SubmitThreadpoolWork API.
These things tend to work better than third-party libraries, and are way easier to consume than developing custom schedulers.
Is OpenMP part of the C++ standard these days?
Assuming it meets your needs it has (AFAIK) support from all the major compilers (Intel, MS, GCC, LLVM) as well as being able to abstract over GPUs and other sorts of hardware.
The level of support varies, but OpenMP 2.0 standard is supported by all 4 of them.
Embedded tends to have their own vendor forks, even if based on GCC/clang, it isn't the same thing.
Then we have game consoles as well.
> * write all intersection and BVH code by yourself;
> * build and use open-source third-party library;
> * use something like Intel Embree.
How many more useful ways are there? Pretty much the only better thing that comes to my mind is "study something like Embree and then write a shader compiler that produces tracing code on-the-fly", and even that perhaps might not be actually better (although it's my understanding that OSL may actually be doing something like this).
I wrote the following programs in an attempt to get a better answer to that question.
http://canonical.org/~kragen/sw/aspmisc/my-very-first-raytra... four pages of C
http://canonical.org/~kragen/sw/dev3/raytracer1k.clj 1K of Clojure, condensed down from
http://canonical.org/~kragen/sw/dev3/circle.clj a page of unfactored but not obfuscated Clojure, translated into
http://canonical.org/~kragen/sw/dev3/circle.js a page of JS in node.js
https://gitlab.com/kragen/bubbleos/blob/master/yeso/sdf.lua a page of Lua using signed distance function raymarching that runs in real time at a lousy framerate; to run it, git clone the repo, install the prerequisites listed in the README, and run `make` in the yeso directory, before running sdf.lua.
But none of these can load triangle meshes, render caustics, render area light sources, average multiple samples, do bidirectional rendering, etc.
Reznik says, "A lot of projects using something like PPM file format, which is very simple, but in the end, you need to write ... extra code for this simplicity (like tone mapping, converting from floating point values to uint8, etc.)," but the amount of extra code required is really very minimal:
/* PPM P6 file format; see <http://netpbm.sourceforge.net/doc/ppm.html> */
static void
output_header(int ww, int hh)
{ printf("P6\n%d %d\n255\n", ww, hh); }
static unsigned char
byte(double dd) { return dd > 1 ? 255 : dd < 0 ? 0 : dd * 255 + 0.5; }
static void
encode_color(color co)
{ putchar(byte(co.x)); putchar(byte(co.y)); putchar(byte(co.z)); }
That also leads to a one-page OCaml version and a two-page C++ version by Jon D. Harrop (the Flying Frog guy), both also using PPM output and rendering only spheres, much like most of my examples: https://web.archive.org/web/20070605173709/http://www.ffcons...
He also wrote an SML version https://web.archive.org/web/20070522030632/http://www.ffcons... and a Java version, and others ported it to Scheme and Common Lisp https://web.archive.org/web/20070515185350/http://www.ffcons..., those these seem to be lost now.
Still others ported it to Io http://mike-austin.com/io/ray.io and Factor https://github.com/factor/factor/blob/master/extra/benchmark... (3 pages).
In https://www.pouet.net/prod.php?which=83222 Holtsetio wrote a raytracer in two pages of MySQL SQL, but it's sort of obfuscated. It emits the output in BMP format (almost as simple as PPM) and uses lots of imperative MySQL extensions to SQL, and it ended up as 10 KiB instead of the 1 KiB of my Clojure version. But it handles triangles, not just spheres, so it can render the Stanford bunny.
https://github.com/chunky/sqlraytracer/blob/master/raytracer... is instead about three pages of SQL, using recursive CTEs instead of imperative assignments, and supporting only spheres but with different materials, like my C version. He's using recursive CTEs as implemented in Postgres: compliant with ANSI SQL but not implemented in most other SQL engines. In particular, he started out with SQLite but it wasn't strong enough.
EDIT: it may look gibberish but it's still a structured source file, that requires (simple) lexical analysis etc
All the rest is pretty uninteresting, in my opinion- the various tricks to make better rendering in shorter times is mostly grad-level math and some computer engineering, while the asset loading is a long solved problem with ultra-mature libraries.
I think my interests are probably pretty different from yours, because I find the details of the graphics system and I/O pretty interesting, in particular how they can be improved. I also think the various tricks for better rendering in shorter time are super interesting, and even the question of how to do liability loading has some very interesting new answers, like FlatBuffers. Probably the liability-loading strategies that we came up with in the previous millennium for spinning rust that transferred 40 megabytes per second after an 8 000 000 nanosecond seek time are not optimal for SSDs that transfer 4000 megabytes per second with a 1000 nanosecond "seek time".
Another thing worth mentioning is the question of how you model the 3-D shapes you want to render in the first place, which includes considerations of HCI, sometimes physics simulation, and so on. One of the most interesting things about SDFs is the tempting possibility of an efficiently renderable representation that naturally supports operations like CSG and filleting (though not usually both at once!)
That is to say, I think the stuff these minimal raytracers leave out is actually very interesting indeed. But I also think it's very interesting to see what's left over when you do leave it out: a relatively small amount of math and code that can still produce visually arresting images.
As for asset loading, like I said, I think it's a solved problem. When I start Cyperpunk 2077 on my machine (fresh, no cache), it takes less than 30 seconds to get to a playable state and my SSD is basically streaming data into RAM as fast as it can.
If you only want diffuse reflection and shadows, which is a thing you could potentially use ray tracing for, you don't even need the specular-reflection law. This might be a sensible thing to do if, for example, you were rendering a 3-D IFS; using a raytracer with BVHs allows you to do this lazily and thus get immensely more detail than a rasterizer could do.
(An alternative to conventional raytracing I've been thinking might be fun is to train a PINN on a scene with the wave equation --- a differentiable neural network which maps (x, y, z) tuples to "electromagnetic field" vectors or just pointwise amplitude and phase, trained using a loss that incorporates its deviation from the wave equation and the given boundary conditions. Then you can evaluate the amplitude of light at every pixel on an imaging plane behind a lens or pinhole camera, thus simulating bokeh, reflection, refraction, defocus, and potentially even diffraction and polarization effects. Then instead of explicitly programming Heron's reflection law, Snell's law, the Fresnel equations, etc., they'd just appear as emergent properties of the wave equation. May not work out in practice tho.)
Perhaps if Cyperpunk 2077 were using a liability pipeline that was rethought with SSDs in mind, you wouldn't have to have all that data in your RAM at once; most of it could be left on the SSD until later, thus avoiding the 30-second delay. (As I suggested, SSDs are maybe two orders of magnitude higher bandwidth, but have nearly four orders of magnitude lower read latency.) This kind of thing is a bigger problem for open-world-type games like Minetest where the data volume is potentially unbounded because players add to it over time, and where the 30-second glitches can't be papered over with cutscenes and the like.
BTW, your parenthetical 3rd paragraph is a fairly deep question and fast approximations to it would be useful beyond just rendering scenes realistically. I assume you're familiar with the use of lenses to implement FTs long before they were computationally tractable (https://en.wikipedia.org/wiki/Fourier_optics)?
The PINN idea seems interesting but it may or may not work out; if you're interested in that kind of thing, there's more of it in Dercuano, Derctuo, and Dernocua. Some of the ideas in there have been tested and work, others have been tested and found not to work, but most are still untested.
As for Fourier optics, yeah, I worked with a guy last millennium who had worked on some optical analog pattern recognition stuff at university. I've never gotten it to work myself; I suck at lab work so far.
Something I've thought would be interesting is simulating the entire light field by some kind of finite element analysis, but where each element uses a neural network to more quickly converge to the correct lightfield -- and, perhaps like that paper, to use neural networks to give a compact representation of the lightfield at that element.
The stratospheric view of what a raytracer even is is a monte carlo sampler to evaluate a (fairly complicated) integral. It seems like many tricks for raytracing are about finding better ways to estimate good sampling distributions (for example, when scattering, a heuristic is to send more rays toward light sources). So, being able to simulate the light field more accurately and quicker could get the rendering integral to converge quicker, too.
It sort of reminds me of game-playing AIs like AlphaGo. There's a tree search: the AI is playing a game where a ray is bouncing in a scene, and whenever the beam is scattering it gets to choose which direction to send it -- its goal is to collect light that gives close to the correct pixel color with the least number of beams.
That being said, while I haven't investigated this super thoroughly, in my experience the delta is usually small.
If I remove the FS operations it's instead 160ms initially, then drops to 130ms by the third go and hovers around there.
You can can even add a fairly basic kind of Reinhard tone mapping, plus a 2.2 gamma and output dithering if you change the second function to:
static unsigned char
byte(double dd) { return pow(dd / (1.0 + dd), 1.0/2.2) * 255.0 + drand48(); }
There's an interesting thing that happens where sometimes if you constrain yourself to 1% or 0.1% of the code you would normally use, you end up getting interesting uncontrolled effects. cf. http://canonical.org/~kragen/bytebeat
On .NET side, I used ImageSharp.
Thanks for sharing the Clojure version of a RayTracer.
I'm halfway through and at 1531 lines in Elixir including the tests via doctest :)