Lentil – Lenses for pathtracers | Hacker News Reader

Lentil – Lenses for pathtracers

lentil.xyz

63 pointscuctas39 comments

I'm not sure where they're getting that "lenses in pathtracers are ignored". They most certainly are not. How would physically based path tracing even work if they were?

e.g. https://blog.demofox.org/2018/07/04/pathtraced-depth-of-fiel...

Traditionally most cg rendering systems use an idealized lens to simplify the math. Matrix projection transformations simulate a pinhole lens.

This does mean that all images are perfectly sharp however, so for depth-of-field, bokeh, and other realistic lens blurring effects the position of the idealized lens (i.e. virtual pinhole) is varied over samples creating a sort of fuzzy pinhole.

Most commercial renderers that I know of just render via simulating a pin-hole camera. Any Camera lens effects, lens flare, bokeh is just simulated off of the very simple pin-hole model. The method in Lentil appears to be actually trying to simulate the ray bounces internal to the camera lenses via the geometry from patent information. I am not an expert on optics, but from what I know about modern lens design are these elements are coated to change reflection and designed to prevent optical problems that don't exist in most computer renders (Chromatic Aberration). My initial take is this simulation incorrectly solves a problem that most users rendering things have no interest in even dealing with. Computer rendering has moved into this simulating the physical world but without asking why, in many rendering cases you do not want a camera that behaves like the physical world, you want it to behave better and with less control.

I think most commercial renderers offer a thin lens model in addition to pin-hole to get DoF and custom bokeh, but using a real multi-lens setup gets you some more realistic effects.

The main use case that popped in my head was compositing of rendered effects with live footage from a physical camera. In this case, might it be beneficial for your render to exactly match the real world lens, defects and all?

It is typically done in compositing since it is impractical to render with depth of field which gives you a lot more noise and takes away your ability to change it without a re-render.

I think it means lens thickness and design imperfection is "ignored". The renderer typcially uses a simple aperture + perfect thin-lens approximation.

The name makes sense if you're aware of the etymology of the word "lentil" [1]:

From Middle English lentile, from Old French lentille from Latin lenticula, diminutive of lēns, from a Proto-Indo-European root shared by German Linse and Ancient Greek λάθυρος (láthuros). Doublet of lenticula.

In Swedish, for example, the word for the food ("lins") is exactly the same as the word for the optical component. I just thought that was interesting, since naming is hard and perhaps the connection was not obvious to everybody.

[1] https://en.wiktionary.org/wiki/lentil

In modern French both 'lentil' and 'lens' are still lentille as well.

Lentil is an advanced camera toolkit for the Arnold renderer. It is a set of shaders that extends creative control and physical correctness, while significantly reducing the time (~30x) needed to render bokeh through bidirectional sampling of camera rays.

Efficiently simulating realistic lens systems has been a thing for a while; the original paper by Kolb, Mitchell, and Hanrahan is here: https://www.cs.utexas.edu/~fussell/courses/cs395t/lens.pdf. In the paper they talk about how to apply importance sampling, which is the key to efficiency in bidirectional path tracing. What does this package offer beyond that?

This seems like misleading advertisement to me.

They claim to be 30x faster than the Arnold CPU renderer, but of course most people use the GPU renderer, which this tool doesn't support and which is potentially 100x faster.

So they are comparing their main speed claim against an outdated and known slow implementation.

Also, vray (another renderer) ships with a variety of lens shaders and if I recall correctly already decouples main rays and camera focus rays.

What I agree on is that having more realistic lens shaders is great and that they should be fast to calculate. But since doing bokeh in post is good enough for Hollywood movies, I'm not sure if their solution offers a large enough increase in image quality to justify the surely large increase in rendering costs.

"real" as in "physically plausible simulation"

I was imagining some kind of really wacky setup where you fire light from a simulation through a real lens to get realistic lens effects. But what kind of display would you need?

Now I am picturing an 8K OLED monitor on rails moving toward a camera.

I think they call them phase addressable full spectrum electromagnetic field emitters in Star Trek.

Why can’t you just instantiate a lens in front of the camera and render rays through it?

Not an expert, I would guess at least these two things:

1. There are actually around 20 lenses in a camera, not one.

2. A lens converts a ray into a cone, right? And then you need to pass that cone through the other 20 lenses which further modify it's shape, differently at different wavelengths.

I would guess this is extremely computationally expensive done naively.

Cone tracing is a thing that exists, but I think most path tracers just just regular rays. (There's also ray differentials, which if I understand correctly, are a way of tracking how a cone of light would spread out after bouncing off of curved surfaces, if the ray were a cone.)

As to why lenses would cause a problem specifically, I don't know the path tracing algorithm well enough to say for sure offhand, but it may have something to do with introducing random sampling before you even hit the first scene object. Usually in ray tracing the first hit is kind of a freebie; you can calculate direct illumination exactly, and it's only indirect illumination that's approximated. (In path tracing, it's approximated by doing a lot of random sampling.) So, not having to approximate the first-hit direct illumination reduces the noise quite a bit right off the bat.

You can. It's just very costly. Rays typically represent just a single sample, so the more probabilistic the projected image becomes (read "unfocused"), the more samples are required to capture it smoothly. This is compounded by the number of lens elements, the material's index of refraction and other complexities. A single element lens adds a minimum of two additional ray/surface evaluations for every sample of which their are many per pixel.

Also, as a separate matter many implementations optimize around the assumption that the projection from three dimensions to two is done in a uniform, undistorted space. For example the kind of projection that can be fully described by a 4x4 matrix. Changing that assumption loses those optimizations, and generally leads to substance abuse and mayhem.

Is unfocused light in real life actually probabilistic?

I thought all light transfer was probabilistic. (You should read Feynman's QED for more)

It would make sense.

Yes. But probabilistic isn't the word I would use.

https://en.wikipedia.org/wiki/Wave%E2%80%93particle_duality

No. Bokeh can be perfectly described with classical optics.

You can at the cost of some light bounces.

Most modern render engines render rays from the camera into the scene. This speeds up rendering a lot but also means rendering caustics will need a huge amount of samples because the probability of seeing light from a camera through glass is very low. Bi-directional path tracing fixes some of these problems.

So it can be done but always at a cost (rendering times).

In Blender I have done this a couple of times and even used different IORs for the RGB colors to get real world results.

Is there any kind of standard file format describing lens internals, or database of different lens designs or actual commercially available lenses? I'm interested in this kind of simulation for some machine vision applications that may need to cope with not all of the image being in perfect focus at once.

I have no Idea what this is for and what this is actually, but the website is beautiful.

Can someone ELI5 on the significance.

https://www.reddit.com/r/explainlikeimfive/comments/9954a9/e...

I think the poster meant an ELI5 on what Lentil brings and how it works compared to others in its class, not what ray tracing is in general.

Path tracing is currently state-of-the-art for making photorealistic CG images because it can generate real reflections, refractions &c. by simulating something very similar to how the physical world works.

A photo (or movie) is, however, taken through a lens that distorts the image (sometimes intentionally; what is and isn't in focus in a composition is important for drawing the viewers attention).

A path tracer can already simulate this by just putting a bunch of pieces of glass shaped like the real lenses of a camera, but this is extremely computationally intensive.

This particular implementation uses a few (already known in the literature) techniques for making it faster: bidirectional tracing, selective oversampling, and thin-lens approximations.

This is an implementation of these techniques for Arnold, which is a renderer implementation used for many feature films[1].

1: https://en.wikipedia.org/wiki/Arnold_(software)

Probably not much - the reality is that rendering depth of field in camera in production CG is a very tough sell. Even if there is no speed penalty, renders are usually so expensive that it isn't even a question of doing it in real time in compositing.

Is the Arnold renderer spectral (or optionally so)?

I thought they did this for the Pixar movie, Wall-E.

Wall E probably was rendered using a server cluster while this might be working with commercial GPUs.

Arnold has both CPU and GPU rendering support. It is used notably by Sony Pictures Imageworks and created my Marcos Fajardo now working at Solid Angle.

Toy Story 4 as well: https://www.youtube.com/watch?v=AcZ2OY5-TeM

Pixar uses RenderMan, which is Arnold's competition.

That video has nothing to do with this. This plug-in is just a sampling strategy to make rendered depth of field less noisy.

The video you linked just talks about a shot that was purposely made to be an homage to a classic cinematic technique even though it isn't necessary. Depth of field in production computer graphics is almost exclusively done with depth maps in compositing. Because there are many different layers that won't give seam artifacts around the edges as well as floating point outputs that don't clip, depth maps are even easier to make work.

Reply on news.ycombinator.com