Hacker News Reader

Comment by dpiers | Hacker News Reader

Two figures that stood out to me were the 1.1 million render hour / day capacity of the farm and the quoted total of 190 million render hours for the film.

That means almost half a year (~172 days) was spent rendering the film on this supercomputer. The listed running time of 108 minutes * 60 seconds * 24 frames per second = 155,520 frames in the film, giving us an average render time of 1,221 compute hours per frame, or a rendering speed of 2.27e^-10 FPS. Which means that, if Moore's Law continues to hold, in 26.7 years or so we'll have a super computer that could render this film in realtime at 24FPS, and that's just neat.

Congrats to everyone who worked on this; really impressive technical achievement.

View on HN

1.2 hours per frame sounds about right for a feature. Also, apart from Moore's Law, there's a Blinn's Law ( http://en.wikipedia.org/wiki/Jim_Blinn ) - As technology advances, rendering time remains constant.

It's 1200, not 1.2 hours. However, there are at least twice as many frames, as it is 3D.

My bad. It has been a dogma for decade(s) now that production quality feature is from 0.5 to 2 hrs per frame for final though, so that was my reaction without reading correctly. You have to factor in numerous passes and production previews which add up. source: am rendering daily.

TV work, on the other hand, can't deal with that kind of time burden, so it's shifting to more novel solution which lately involves near real-time rendering (see element3d for example).

In my experience, typical render times are whatever can get finished overnight. So up to 8 hours or so on average. Many complicated renders will take much, much longer. And then of course many simple things like previews and non-final renders go significantly faster. It varies widely depending in the complexity of geometry and lighting model.

Blinns law is completely true. The limiting factor on render times is the impatience and work schedule of the people submitting the jobs.

Remember that you do not get it right immediately, thus most sequences are repeatedly re-rendered until everything is correct. Also a frame often isn't rendered in a single pass but is composed of multiple layers -- although this varies depending on the animation studio and their philosophy.

Isn't this layering what they referred as "time consuming manual lightning"? From what I understood, everything is treated as global illumination and effects are done without "tricks".

No, layering just means generating multiple renders for a given shot, which can then be combined to create the final image using 2d compositing software such as Nuke. Full 3d renders are always going to be much slower than just doing image manipulations in 2d, so it is much more efficient to iterate by making adjustments in 2d than by re-rendering everything from scratch every time you need to make a change. Additionally if just one element in a shot changes, say a character's animation changes, it much more efficient to just render the layer for that element again than to re-render ALL the geometry for the entire shot. Each layer may still be rendered with full global illumination and whatever other fancy lighting is needed.. the layering just makes it easier to iterate in individual pieces of a shot.

In feature animation things are changing and being adjusted constantly. Each shot is worked on by many people across multiple disciplines, all of whom do many, many iterations to hone in in the final shot. So the entire production pipeline is designed to make handling changes as efficient and easy as possible. Look at the amount of processing resources disney used for this movie.. I can guarantee you they are cutting all the corners they can and maximizing efficiency wherever possible. Doing a single monolithic render every time something changed would be extremely inefficient.

That said, there are a few studios that do produce complete final frames in-render. Usually places that write their own renderers in-house and therefore have a sort of macho academic attachment to showing off how much their renderer can achieve out of the box. Blue Sky is this way because their whole pipeline is designed around their cg studio renderer. Anecdotally I have heard that Pixar did everything in renderman for a long time but more recently they have started using more of a compositing workflow since it's so much more efficient. So it's possible that disney may be this way now since they have this fancy new hyperion renderer they wrote, but I certainly wouldn't bet on final frames all being done in-render.

But the layers aren't independent at all, so how is it done? For each small change you show have to resolve the full rendering equation (although that should be easier given small changes). So I understand they may want to use something like that for quickly visualizing changes, but I can't see it used for actual rendering.

Each layer might only have some subset of the objects in the scene, but the lighting on those objects might still use the indirect contribution from all other scene objects.

So for example if your scene has objects A, B and C, you might break it up into a layer for each. Then to render the A layer, your primary rays would intersect with object A only. But any secondary/indirect rays would intersect all of A, B and C, so you'd still get the correct global illumination on object A. Breaking it up this way just makes things easier to adjust in compositing. Also the indirect for each layer often will be rendered as a separate pass entirely so it can be dialed in comp independently, or regenerated if the scene geometry changes.

It's also important to understand that almost all the lighting in feature rendering is incredibly faked and not physically-correct at all. This is particularly true in animated movies. It's just important that the end product looks plausible, not that it's academically correct.

Global illumination helps add a bit of realism and nuance, but its contribution to the final image is pretty subtle, especially between objects that are far apart. Nobody would ever notice if the indirect lighting wasn't perfectly correct in all but the most close-together objects. So a lot of the time it won't be re-rendered if the scene changes slightly, as long as it still looks decent.

Without global illumination, to get the illusion of light bouncing between objects lighters would have to place spot lights for every bounce they want to fake. Like a ground light pointing upwards below a character to fake light bouncing off the ground etc. People really used to do this, and it's a real pain. This is the "time consuming manual lightning" they talk about. Global illumination tools make that process automatic and are able to get much richer light interactions than anyone could set up by hand. But they're still just tools. And like any tool, artists will break them apart and use them in whatever hacky way they need to get a shot to look right. Getting the exactly correct solution to the rendering equation is just not important.

It's not like they create the film and then push render though. I assume these figures include massive amount of iterating in different work stages. So if you are interested in only rendering the end product you will get there a lot faster.

Hope someone knowledgeable can comment on this. The impression from the short making-of documentaries packaged with Pixar's films, is that they're completely animated, voiced and viewable with very basic working rendering, and the proper render is something that in principle could be done as a final complete step.

Of course, that's probably a bit of movie magic; in the real world there's surely some iteration.

Yeah, previs (pre-visualisation) is generally always done these days, as it lets the director preview what things "look" like, but that only has very basic geometry, low res textures (if any) and no proper cloth/skinning deformation animation, and no proper shaders / lighting. So it's basically using a game engine without even game engine decent lighting / shading for the most part.

So while it's great for placing objects in the scene and getting the camera position correct, that's about it. There has been a move to using progressive raytracing more and more over the past year or so, but it's still very early days with that.

On top of that, as previs is almost always done early (before most asset generation - modelling, texturing, maybe animation too) is done, it doesn't always give you all the info you need - i.e. it's often the case with mechanical hero objects in the scene (i.e. aircraft, robots, weapons, etc) that the basic geometry used for previs isn't good enough, and later on in real lookdev/lighting (or sometimes even at animation stage) serious issues are found - i.e. the previs has a robot moving through a street, but when animation actually start trying to rig the robot to get it moving realistically, they find for the arms to swing, it can't actually fit in the street.

So there's a huge amount of iteration that works up-and-down the pipeline, sometimes causing lots of different parts of it to re-do work.

Generally it goes: Previs -> Modelling / Texturing (linked, as you need UVs) -> Layout -> Animation - > Lookdev / Lighting -> Compositing.

Any non-trivial change before Lookdev / Lighting will trigger new renders having to be done for some layers / scenes.

The above poster is right. It's not just some iteration, there is tremendous iteration. The quality that you see comes from iterating until there are no more imperfections and that takes a lot of iterations and a lot of renders. A typical lighter may have 4 shots they are working on at one time, with one finishing about once a week.

So basically the impression from the DVDs is a simplistic fantasy. While the narration says 'it's such a wonderful place' the people working there until 9 every night are thinking 'why is everything broken'.

There's a great deal of iteration (look development, lighting) past animation approval.

1.1 million render hours per day on 55000 cores means that each core in their render farm is rendering only 22 hours per day.

With that many cores, would there be some expected down time?

i doubt it was rendered at 24fps because it probably is a 3D movie.

You're right; the frame count should be doubled, and the other calculations adjusted accordingly.

I can't find the source, but I remember reading that when Pixar went back to render Tory Story in 3D that they got close to 24 fps, just because the rendering hardware and software had improved so much.

But without a source, I'm not going to stand by that...

Moore's Law is about transistor density, it says nothing about performance.

Well, performance scaling in software has advanced (in average) with a stronger exponent than Moore's Law. Which is why we can get visuals as awesome as Jurassic Park on modern GPUs - while the hardware isn't quite as powerful as the rendering farms used for that movie, other advancements allow us to achieve what are in many ways more spectacular results.

So, he may be wrong about what makes this happen, in essence he's probably correct. In the time frame he outlined, I expect both Ray Tracing and Radiosity solutions on both hardware and software to match and possibly exceed what he has outlined in terms of capability.

"Moore's Law" as a phrase in popular usage dates from way after Moore's 1965 paper on transistor density doubling every year, and anyways Moore wrote a memo endorsing it's usage for things other than that (It was included in the lecture notes of some semiconductor physics class I was taking).

Increased density = more transistors. Don't be pedantic.

And more transistors doesn't mean more performance. Why do you think modern CPUs have 6 or 8 cores on? Because we can fit more transistors on a die but CANNOT make individual cores go any faster.

You can massively parallelize rendering a movie in advance because you can do each frame on its own CPU. Rendering in real-time is much less easy to extract this kind of parallelism from, particularly if you have hard real time constraints.

I thought the problem with faster as you go smaller is that smaller pipes leak higher pressure water (electron tunneling), which is why we have divided the pressure into a larger number of small pipes (multi-core).

If we can make higher pressure capable small pipes we could run then all faster. I bet there's billions being invested in solving this problem, but that doesn't mean it's solvable.

One of the main limitations is actually power, as increasing the frequency increases the power draw. All that power turns into heat, which has to be dissipated. Modern processors already shut off inactive cores and change the frequency of the active cores on the fly depending on the workload to reduce the power draw [1], so it's easier to add performance by adding cores. The problem is that software hasn't quite caught up to take advantage of the added cores.

[1] http://en.wikipedia.org/wiki/CPU_power_dissipation

Modern CPU cores are faster than CPU cores from before the "multicore" era. I think we're figuring out how to make CPU cores faster _and_ pack many of them into the same space.

I was with you until you made a 26.7 year prediction.

Reply on news.ycombinator.com