Pixar's Chris Horne Sheds New Light on Monsters University
thisanimatedlife.blogspot.com
thisanimatedlife.blogspot.com
First, you have to understand the cheaper alternative, which is called local illumination. With local illumination, for each pixel, you figure out what object you're looking at, and where on that object. You take into account the normal (direction of the surface at that point) and the optical properties of the object at that point. You also take into account the position, intensity, color, etc. of any light sources in the scene. Optionally, you may also take into account any shadow casting. That's it.
What's missing from that list? It's a big one: You're not taking into account the way other objects in the scene affect that little point. In the real world, light bounces all around. Each little point is affected by pretty much each other little point. All the points are interdependent.
But with local illumination, you ignore the way other surfaces contribute to the point's illumination. You're just looking at that one point and the light sources. That's why it's called local.
Global illumination, by contrast, does take into account the interplay between different points in the scene. Its main purpose is to simulate light bouncing between polygons.
As you can imagine, managing the complexity of all those interactions is a tall order. We have quite a few algorithms for this; all are approximations. It's worth noting that some of these approximations can converge towards a provably physically correct result if you let them run long enough.
In any case, running global illumination often causes a major increase in rendering time. So it's understandable that Pixar, which has to render a huge number of frames at huge resolutions, did not traditionally use it much.
Do we have any ball-park estimate of how long it takes Pixar to render a single frame of a movie like Monsters U?
EDIT: Many people are mentioning it's done massively parallel, which I meant to include in my question. So, what I mean is, how long does it take to render a whole Pixar movie?
http://news.cnet.com/8301-13772_3-20068109-52/new-technology...
[1] http://www.slashfilm.com/cool-stuff-a-look-at-pixar-and-luca... [2] http://jalopnik.com/5813587/12500-cpu-cores-were-required-to...
It sounds a bit like the n-body problem, which has parallel approximation algorithms, but nothing terribly straightforward.
But I'm just guessing about all this.
In another way of thinking about it, raytracing simulates photons. Photons don't interact with each other, so the problem of simulating photons is massively parallel.
So you don't have to simulate the same photons lots of times for different parts of the frame. You just simulate the photons that will eventually end up in a certain part of the image.
There are a lot of other things going into each scene other than just the rendering (storyboarding, animating, lighting, etc) so it would be silly to wait for the entire film to be done and then render all at once.
Also the 11.5 hours is the average time. Single frames could take up to 80 or 90 hours to render according to that article.
So as for how many they do in parallel that would be very interesting to find out, but I would guess possibly all the frames devoted to a single scene so that would be around 5,000-10,000 frames at once which seems somewhat reasonable considering they have 12,500 cores.
There's also another factor at play, which is directability. Physical correctness is not usually a priority except as far as it advances the artistic goals of the people making the movie. If the director says, "can you make the right side of that table look less red?", you need to have some way for the artist to achieve that goal, even if that's not how the scene would "really" look. I expect that the development of new tools and processes to allow precise manipulation of the lighting in globally illuminated scenes was just as much, if not more, of a barrier than the additional cost in rendering time.
Less flexible, more scripted behavior is often the smarter choice when you want to be able to ensure a certain gameplay experience.
And the resulting primary gameplay experience is boredom; felt most heavily recently with Bioshock Infinite.
The other type of game is the open world formula, featured in Assassin's Creed and GTA, and to a certain extent Fallout, Skyrim etc. But these become boring in another way; they rely on making navigating the territory interesting, but eventually the novelty wears off and you just want to enable the "instant teleport" function.
I still miss games like Thief, where navigating the territory was the main challenge of the game, but the territory was carefully enough designed, yet still very open, and not seen repeatedly enough to become boring. Dishonored came within 60%, but the player character was too powerful.
To that list i'll add system shock 2 and Dues Ex 1
Team Ico games come close too.
We used GI at pixar when it was appropriate, even at the expense of long render times - that is to say, only when it made the final product look better. How you get to the result doesn't matter, only what it looks like on screen.
Pixar have had a global illumination system in place at least since Up, and maybe earlier [1]. However, it was one that integrated with their rasterizer.
The article is now claiming that Pixar have switched to Raytracing exclusively, which really is actually a HUGE change, as Renderman only introduced raytracing at all with Cars 2. Every prior Pixar movie exclusively used a micropolygon rasterizer for rendering.
The article also claims:
> ray tracing is a relatively advanced CG lighting technique
Well, not really. Ray tracing - at least Whitted-style ray tracing - is about as simple as physically-based rendering gets. It's making it fast that gets complex, but it's possible to write a basic ray tracer in a few hours if you know what you're doing.
[1] http://graphics.pixar.com/library/PointBasedGlobalIlluminati...
Not really correct. Ray tracing is more like the most simple GC lightning technique you could imagine, but so incredibly computationally expensive people have been mostly waiting for the hardware to be good enough for the last 50 years.
And in the meantime, they've been using an incredible pile of hack and tricks to try and approach levels of visual quality and complexity trivial on a raytracer, except said pile of hack could actually be computed before the heat death of the universe.
I was surprised that ray tracing in Pixar
was historically a clunky, haphazard process.
I always thought of it as this smooth, polished
machine like something you would see at an Apple store.
I'm going to have to look more critically at Pixar movies now, knowing that the old ones didn't have actual calculated light sources.
Nevertheless we often faked that with massive light counts. For the opening shot of Armageddon with the astroid hitting earth I used 20,000 point lights to represent secondary debris reentering the atmosphere. In 1998 or when ever that was it still only took RenderMan a few minutes to render each frame.
They went through a phase of hiring hip young things fresh out of MIT to write tools. Instead of nice friendly tools that play well together, they got a lot of domain specific languages.
Even the commercial stuff looks like some graduate students thesis work. A basic Java GUI and about 50+ commandline arguments.
What's new with MU is they're using both physically-plausible shading (where the shading is based off physically-based BRDF lighting algorithms, which gives much more realistic results), and global illumination path-tracing for the entire light transfer equation.
I usec renderman in the mid 90s and it allowed selective raytracing per shader.
Certaingly the guts at cgsociety think there was raytracing in both Toy Story and A Bug's Life.
For PRMan 13 (which Pixar used for Cars in 2006), they added semi-decent acceleration structures which sped up raytracing a bit. But you still had to use custom shaders to cast rays.
With PRMan 17, ray tracing is now a first-class citizen in PRMan, and it can also trace rays from the camera instead of doing the traditional (pre 17) REYES rasterization of the surface and then shading that surface for reflection based on ray tracing.
I'm sure this update is significant, and sounds like a ground up reworking of the engine, perhaps replacing REYES? But it's not at all accurate to say the Pixar is moving to raytracing. Pixar has been in that neighborhood for more then a decade.
Source: Apodaca,Gritz, Advanced Renderman -- MK 2000.
http://www.bigscreenanimation.com/2008/09/toy-story-re-relea...
Were these re-rendered versions released in 2D on Blu-ray?
The contrast to Brave isn't that extreme.
Hence, it's not a realtime "voxel engine" as far as visual rendering goes.
Pixar probably didn't need to do a voxel -> triangle transformation to their data set if they were using voxels, but any real-time renderer does.
There are a lot of very nice mathematical properties of triangles, and the trend towards graphical fidelity has really only served to balloon budgets in the gaming space. We don't need voxels, and honestly neither do we need ray tracing.
It's not even new tech--games going back to Outcast, some Build engine games, Novalogic stuff, and so on have used voxels. Ray tracing has been used in a handful of nifty tech demos, but otherwise nobody cares.
Ray tracing and voxel tech is of marginal utility for games, and things have moved on--it'd be like switching the US construction industry over to metric; too late to make a difference and too minor to matter.
EDIT: In spite of all this, voxel cone tracing looks sexy as hell though.
There are voxel engines now that store world data using voxels while what the player actually sees is polygons based on that voxel data.
I'm reminded of one guy's side project[1] that accomplishes exactly that.
[1]http://procworld.blogspot.com/2012/12/videos-of-caves-and-bu...
Also don't forget voxels take an order of magnitude more memory. Voxels are hyped because of minecraft, and in terms of acceleration they're really not trivial at all.
I remember some nifty demo with voxels though. But honestly wait for the real-time graphics API mess to smooth up a little before going voxels. You can still do a big lot with triangles.
Are they going all the way to an unbiased global light transport algorithm (like LuxRender) or just using basic raytracing (like PovRay)?
Are they using an existing renderer? If not, are they releasing their own like they did with RenderMan?
Are they rendering with CPUs or with GPUs?
How much time per frame does it take them with how many cores of what sort?
Our entire function in the filmmaking business is to tell a story visually, and for that you need complete control and directability of the image. This is the opposite goal of unbiased renderers. Nevertheless more tools in the tool box is always good.
Biased generally means it's interpolated with a point or irradiance cache.
PRMan 17's a fairly good raytracer now (Arnold was giving PRMan a bit of a kicking in this department over the last two years).
All on the CPU - there's no way GPUs can cope with the size of textures and geometry feature films need to cope with (up to 200 GB of textures and Geometry in some of the complicated scenes) - there's no way that's fitting on a GPU.
I don't know what Pixar have, but SPI (who use their own version of Arnold) used to have quad socket i7s, so 64-thread machines with 96 GB of RAM two years ago - some of the more complex frames were taking +30 hours at 2k.
BDPT is more concerned with the surface area of meshes and solid angles of hits, so that the light path vertex weightings can be accurate.
Having said that, what renderman is and what pixar animation do/use are rather orthogonal. For example pixar made heavy use of Subsurface scattering in the incredibles. Something that was at the time rather time intensive.
The title is a bit misleading as they've been using raytracing for years 1
[1]http://graphics.pixar.com/library/RayTracingCars/paper.pdf
I don't know much about graphics but maybe some of you will find it illuminating (no pun intended....)
REYES-rendered scenes that fake global illumination are pretty arcanely hacked together. Just making a legacy scene ray-traced would make it look worse, not better.
If I can make my webapp better (however you define "better") than my competitors' using PHP and MySQL, while they're making theirs using Ruby on Rails, MongoDB, etc,etc. Does the tech stack in the background matter, aside from making a nice article?
I think it does, because the tech stack in the background allows for things that might not be possible for other tech stacks.
You can duplicate somebody else's webapp in your backend of choice, but you can't have true GI if your rendering engine doesn't support it, and while you can fake some of the effects, they ultimately won't look as good as the real thing (unless you're aiming for a different 'good').
There's the obvious render time, but actually render time isn't that important - studios are happy to wait up to 30 hours for a 4k frame on the farm if that's what it takes for a shot. But they don't want artists waiting around, so they want very quick iterations and previews of what the artists are doing, as it's the artists who cost money.
This is why global illumination has taken off over the last 5/6 years (thanks largely to SPI and Bluesky showing it could be done), as although the render times are slower, it means lighting the scene (by phyiscally-based shading) is much quicker and you don't need as many hacks as you did with PRMan (light maps, shadow maps, reflection maps, point caches, etc). You can literally model scenes with lights as they are in the real world.
On top of this, there's how easy it is to do very complex shots and change just bits of it - tools like Katana allow hugely complex scenes to be managed and rendered very efficiently, with very little work from artists. Studios who don't have similar tools often duplicate and waste a lot of time doing things that should be easy.
For example, Weta on IronMan 3 wasted a lot of time doing all the different suits, as they didn't have a decent asset-based pipeline that would have allowed them to re-use a lot of shaders, assets for each suit.
So while I'd say they are still way ahead in efficacy they are at par or behind with respect to light simulation.
SPI have been using full GI pathtracing with Arnold renderer for the last 5 years, and Blue Sky (studio which did Ice Age) has their own GI raytracer they use as well.
Also, Pixar don't actually have that big a renderfarm - they don't need it. Other places like Weta and ILM have renderfarms that are much bigger, but are used for multiple productions at once, and for doing things like compositing and fluid/cloth/physics sims.
It's only recently that advances in processing power have made physically-based ray tracing practical for film production - particularly with the take-up of the Arnold renderer by various other studios. Suddenly lighting becomes a matter of placing lights and letting the computer do the work rather than needing to carefully set up the correct impression of light in the way a painter might. So it requires quite a bit of change of approach from the artist, and you can imagine why there'd be a bit of a cultural problem introducing this.
That said, of the magic comes from a toolchain that lets artists work with curved surfaces (NURBS or similar) and that converts those surfaces to polygons at the last minute. We finally started getting hardware support for that sort of thing with DirectX 11 and OpenGL 4.