Hacker News Reader

Comment by akritrime | Hacker News Reader

Hmm, makes sense. Does same principles also hold while rendering the movie? The 80/20 rule will be harder to maintain because assets can change frequently and unexpectedly during production. But I guess the need for the data to geographically replicated, with fast and easy access, remains (especially during these remote times). During those times, is s3 a still good enough choice or do you want something more robust so your rendering process is not bottlenecked by io.

View on HN

Rendering is different because the assets are very interdependent and the tooling is very POSIX specific. Rendering a scene (especially with ray tracing) can use thousands of assets from the movie even if they aren't directly on screen. Artists draw and save files in tools like Autodesk Maya and renderers read files from disk and work on them while in memory.

For some assets and tooling FUSE mounted s3 can work but generally FUSE and other userspace mounters I've seen slow down artists and rendering measurably.

Think of rendering assets more like code files and git with trunk based development. You want all of your artists to use the latest assets which are daily being updated. All of the assets should be co-located and you don't want geo-replication because of latency. Even if your artists are located all over the world you'll want them to store the saved assets in one place. Where the rendering happens.

There will be different assets that are hot as the movie progresses. But you're more likely to try to keep the latest version of all assets hot rather than all versions of specific assets hot.

Most studios use HPC style environments. NFS + big compute servers connected with high bandwidth.

Interesting. Is Kubernetes ever used as the orchestrator in HPC workloads, or rather are even containers used there? Also what are some good resources to get better at kubernetes? Currently I am mostly playing with managed k8s like digital ocean's and thinking about transitioning to k3s based bare-metal solution. So, I would really like to learn what's the right way of doing this.

There are some HPC environments that use Kubernetes but they likely use custom schedulers optimized for batch workloads (e.g. https://github.com/volcano-sh/volcano).

"containers" are often used but not always docker containers. HPC environments I've seen will often use container primitives (e.g. cgroups, namespaces).

There's a lot you can learn with managed Kubernetes and it's a great place to start. You can learn a lot of the parts of Kubernetes with running through https://github.com/kelseyhightower/kubernetes-the-hard-way or reading https://www.amazon.com/Kubernetes-Running-Dive-Future-Infras...

I'll email you to follow-up since tracking HN comments isn't a great way to have a conversation.

Reply on news.ycombinator.com