Simulating fluids, fire, and smoke in real-time
andrewkchan.dev
andrewkchan.dev
Also, in industrial CFD, where the Reynolds numbers are higher you'd never want something like counteracting artificial dissipation of the numerical method by trying to applying noise. In fact, quite often people want artificial dissipation to stabilize high Re simulations! Guess the requirements in computer graphics are more inclined towards making something that looks right instead of getting the physics right.
That is exactly correct. That said, as something of a physics nerd (it was a big part of the EE curriculum in school) people often chuckled at me in the arcade pointing out things which were violating various principles of physics :-). And one of the fun things was listening to a translated interview with the physicist who worked on Mario World at Nintendo who stressed that while the physics of Mario's world were not the same as "real" physics, they were consistent and had rules just like real physics did, and how that was important for players to understand what they could and could not do in the game (and how they might solve a puzzle in the game).
Reactions are an important part of this, it's not just how good an actor is, but how other character react to them. If they are ignored, it's like that character doesn't exist, has no effect, is inconsequential, doesn't matter... is not real.
In a fluid simulation, when water does not react to the shoreline, rocks in the water, columns supporting a pier etc, it is like the water (or the obstacles) don't exist. In this sense, the water (or the obstacles) "aren't real".
For example, the water in Sea of Thieves looks magnificent... but because it doesn't interact (it's procedural), it doesn't feel real. It isn't real.
A further problem here with water is that it's difficult to make a continuous fluid that is consistent and isn't realistic, because the basic principles of fluid are so very very simple: conservation of mass, conservation of momentum.
[ Of course, you can still vary density, gravity. There are other effects beyond these: viscosity, surface tension. Also, compressible fluids, different reynolds numbers, adiabatic effects, behaviour at non-everyday temperatures, pressures, scales, velocities, accelerations etc. ]
There are also simplifications, like depth-averaging (Shallow Water Equations aka St Venant), but again, it's hard to vary it and yet remain self-consistent.
Cellular methods - similar to conway's game of life but for fluid - maybe are an exception, because thet are self-consistent - but pretty far from "water", because not lack momentum (and aren't continuous).
The final issue is error: simulations are never perfectly self-consistent anyway, because they must discretize the continuous, which introduces error. In engineering, you can reduce this error with a finer grid, until it too small to matter for your specific application. For computer graphics, it only needs to be perceivably" self-consistent - and perhaps pixel size is one measure of this, for the appearance* of consistency, in area/volume, acceleration, velocity, displacement (...though, we can perceive sub-pixel displacement, e.g. an aliased line.)
That said, never do a PhD for economic reasons alone. It's a period in your life where you are given an opportunity to build up any idea you would like. I enjoyed that aspect very much and hence do not regret it one bit.
On the other hand, I also found out that academia sucks so I now work in a completely different field. Unfortunately it's very difficult to find out whether you'll like academia short of doing a PhD, so you should always go into it with a plan B in the back of your head.
I also found that I don't want to be a good academic, but a good researcher. Most importantly, that these are two very different things.
Oh no. It is also quite possible to do things, that will very firmly close doors, that were open before, or making sure some doors will never open ..
(quite some doors are also not worth going through)
The two sibling comments both mention the economics isn’t guaranteed - and of course nothing is guaranteed on a case by case basis. However, you should be aware that people with PhDs in the US earn an average of around 1.5x more than people with bachelor’s degrees, and the income average for advanced degrees is around 3x higher than people without a 4-year degree. Patents are pretty well dominated by people with PhDs. There’s lots of stats on all this, but I learned it specifically from a report by the St Louis Fed that examined both the incomes as well as the savings rates for people with different educational attainment levels, and also looked at the changing trends over the years, and the differences by race. Statistically speaking, the economics absolutely favor getting an advanced degree of some kind, whether it’s a PhD or MD or JD or whatever.
I've used the basic idea from that paper to make a surprisingly decent program to create gas-giant planet textures: https://github.com/smcameron/gaseous-giganticus
it's a painting program where the paint can be moved around with a similar fluid simulation
We liked to call it “curly noise” when we first did it, and I used it on some shots and shared it with other effects animators at DreamWorks at the time. Actually the very first name we used was “curl of noise” because the implementation was literally curl(noise(x)), but curly noise sounded better/cuter. Curly noise is neat because it’s static and analytic, so you can do a fluid-like looking animation sequence with every frame independently. You don’t need to simulate frame 99 in order to render frame 100, you can send all your frames to the farm to render independently. On the other hand, one thing that’s funny about curly noise is that it’s way more expensive to evaluate at a point in space than a voxel grid fluid update step, at least when using Perlin noise which is what I started with. (Curly noise was cheaper than using PDI’s (Nick Foster’s) production fluid solver at the time, but I think the Stam semi-Lagrangian advection thing started making it’s way around and generally changed things soon after that.)
BTW gaseous giganticus looks really neat! I browsed the simplex noise code for a minute and it looks gnarly, maybe more expensive than Perlin even?
In 2014 when I wrote it, 3D Perlin noise was still patent encumbered. Luckily at the same time I was working on it, reddit user KdotJPG posted a Java implementation of his Open Simplex noise algorithm on r/proceduralgeneration (different than Ken Perlin's Simplex noise), and I ported that to C. But yeah, I think Perlin is a little faster to compute. I think the patent expired just last year.
Also Jan Wedekind recently implemented something pretty similar to gaseous-giganticus, except instead of doing it on the CPU like I did, managed to get it onto the GPU, described here: https://www.wedesoft.de/software/2023/03/20/procedural-globa...
The first rule of real-time computer graphics has essentially always been "Cheat as much as you can get away with (and usually, even if you can't)." Also, it doesn't even have to look right, it just has to look cool! =)
I'm not saying this just to be pedantic - my point is that some people do want some games to have very high levels of realism.
The reason to cheat is that we currently don't have the hardware or software techniques to physically simulate a virtual world in real-time.
The refrain is: Plausible. Not realistic.
Doesn't have to be correct. Just needs to lead the audience to accept it.
Yeah. And, cheat as much as you can. And, then just a little more ;)
And, yes, I've gone way too far down this road in the past.
Also light is REALLY complicated when you get close to a surface. A light simulation that properly handles refraction, diffraction, elastic and inelastic scattering, and anisotropic material properties would be very difficult to build and run. It’s much easier to use material values found from experimental results.
Also, light may be simple but light interaction with electrons (ie. matter) is a very different story!
I just googled and there’s a ton of different papers on doing diffraction in the last 20 years, more that I expected. I watched a talk on this particular one last year: https://ssteinberg.xyz/2023/03/27/rtplt/
Also QED is still a model. If you want to simulate every photon action, might as well build a universe.
Our universe has a surprising amount of detail. We can't even simulate the simplest molecular interactions fully. Even a collision of two hydrogen atoms is too hard - time resolution and space resolution is insanely high, if not infinite.
Afaik you can't even simulate double-slit in high-end offline VFX renderers without custom programming
There are other effects (most notable volume scattering), which could be significant for the rendered image and which are simulatable with raytracing, but are usually neglected for various reasons, often because they are computationally expensive.
Till then, I have some hope, that native support for raytracing on the GPU will allow for more possibilities ..
Like, watch this video: https://www.youtube.com/watch?v=9XWxsJKpYYI
This is a whole story about how the liquids in the bottles ‘isn’t really there’ and how it’s not a ‘real physics simulation’ - all just completely ignoring that none of this is real.
There is a sense in which the bottles in half-life Alyx are ‘fake’ - that they sort of have a magic painting on the outside of them that makes them look like they’re full of liquid and that they’re transparent. But there’s also a sense in which the bottles are real and the world outside them is fake. And another sense in which it’s all just tricks to decide what pixels should be what color 90 times a second.
Clearly, there's some sort of a physics simulation going on there, preserving the volume, some momentum, and taking gravity into account. That the result is being rendered over the shader pipeline rather than the triangle one doesn't make it any more or less "real" than the rest of the game. It's a lie only if the entire game is a lie.
So the non-generated frames are the "fake frames".
https://github.com/darklegion/tremulous/blob/master/src/game...
The problem is my analysis is very weak. My knowledge about linear algebra, differential equations, numerical methods, and so on is approximately limited at the level of an introductory university course. What would you suggest as a good start?
I like reading, but I also like practical exercises. The books I tried to read before to get into CFD were lacking in practical exercises, and when I tried to invent my own exercises, I didn't manage to adapt them to my current level of knowledge, so they were either too hard or too easy. (Consequently, they didn't advance my understanding much.)
For me that's the hard part (which I still don't get).
You could start with Saint Venant equations, although they look complicated they're actually within reach. But you'll have to understand the physics behind first (conservation of mass, of quantity of movement, etc.)
With respect to 2, the standard industry practice is a mesh convergence study and comparing your solver's output to experimental data. Sadly, especially with Reynolds-averaged Navier Stokes, there is no guarantee you'll get a physically correct solution.
https://www.sciencedirect.com/book/9780750665940/numerical-c...
Or, via an amazon link: https://www.amazon.com/Numerical-Computation-Internal-Extern...
https://en.wikipedia.org/wiki/John_Steinhoff
https://en.wikipedia.org/wiki/Vorticity_confinement
And some papers:
https://www.researchgate.net/publication/239547604_Modificat...
https://www.researchgate.net/publication/265066926_Computati...
You just set each pixel’s brightness to be the average brightness of the immediately adjacent pixels. Calculate from bottom to top.
Add a few “hot” pixels moving back and forth along the bottom and boom, instant fire.
Looks very cool for a tiny amount of code and no calculus. :)
"set each pixel’s brightness to be the average brightness of the immediately adjacent pixels" sounds like a convolution ;)
You've been doing calculus the whole time. There's a difference between knowing the path, and walking the path.
Here's a 3Blue1Brown video with intuitive graphics on it https://youtube.com/watch?v=ToIXSwZ1pJU
Can this stuff run on an iGPU while the dGPU is doing more rendering-related tasks? Or are iGPUs just too weak, better to fall all the way down to the CPU.
Almost Minecraft, but with rocket launchers on mars
My guess is that the load will become more and more shared between local and remote computing resources.
To my knowledge though it's very rare for software to take advantage of this.
The GPU is the steam engine hurtling forward, the CPU is just the person shoveling coal into the furnace.
Using the integrated GPU heats up the main die where the CPU is because they live together on the same chip. The die heats up, CPU thermal throttles, CPU stops efficiently feeding data to the GPU at max speed, GPU slows down from under utilization.
In high performance scenarios, the integrated GPU is often a waste of thermal budget.
Assuming cooling really is inadequate for running both the CPU cores and the integrated GPU: for GPU-friendly workloads (i.e. no GPU-unfriendly preprocessing operations for the CPU) it would surely make more sense to use the integrated GPU rather than spend the thermal budget having the CPU cores do that work.
I've seen thermal throttling happening at 60°C because overall, the chip is cool, but one core is maxing or two cores are maxing. Which is common in game dev with your primary thread feeding a GPU with command buffer queues and another scheduling the main game loop.
Even when water cooled or the high end air cooling on my server blades, I see that long term, the system just hits a trade-off point of ~60-70°C and ~85% max CPU clock even when the cooling system is industry grade, loud as hell, and has an HVAC unit backing it. Probably part of why scale out is so popular to distribute load.
When I give real work to any iGPU's on these systems, I see the temps bump 5-10°C and clocks on the CPU cores drop a bit. Could be drivers, could be temp curves, I would think these fancy cooling systems I'm running are performing well though. shrug
Short answer: No, it's not "already busy". GPU's are so powerful now that you can do physics, fancy render passes, fluid sims, "Game AI" unit pathing, and more, at 100+ FPS.
Long answer: You have a "frame budget" which is the amount of time between rendering the super fast "slide show" of frames at 60+ FPS. This gives you between 5 and 30 ms to do a bunch of computation to get the results you need to compute state and render the next frame.
That could be moving units around a map, calculating fire physics, blitting terrain textures, rendering verts with materials. In many game engines, you will see a GPU doing dozens of these separate computations per frame.
GPU's are basically just a secondary computer attached to your main computer. You give it a bunch of jobs to do every frame and it outputs the results. You combine results into something that looks like a game.
> Can this stuff run on an iGPU while the dGPU is doing more rendering-related tasks?
Almost no one is using the iGPU for anything. It's completely ignored because it's usually completely useless compared to your main discrete GPU.
Yes. The GPU is doing most of the work in a lot of modern games.
It isn't great at everything though, and there are limitations due to its architecture being structured almost solely for the purpose of computing massively parallel instructions.
> when do we start to wonder if the GPU can “offload” anything to the whole computer that is hanging off of it
The main bottleneck for speed on most teams is not having enough "GPU devs" to move stuff off the CPU and onto the GPU. Many games suffer in performance due to folks not knowing how to use the GPU properly.
Because of this, nVidia/AMD invest heavily in making general purpose compute easier and easier on the GPU. The successes they have had in doing this over the last decade are nothing less than staggering.
Ultimately, the way it's looking, GPU's are trying to become good at everything the CPU does and then some. We already have modern cloud server architectures that are 90% GPU and 10% CPU as a complete SoC.
Eventually, the CPU may cease to exist entirely as its fundamental design becomes obsolete. This is usually called a GPGPU in modern server infrastructure.
What I've seen shows me that nVidia is working very hard to eliminate this gap though. General purpose computing on the GPU has never been easier, and it gets better every year.
In my opinion, it's only a matter of time before we can run anything we want on the GPU and realize various speed gains.
As for where the 90/10 comes from, it's from the emerging architectures for advanced AI/graphics compute like the DGX H100 [0].
Then the core count stopped increasing too -- except only if you look in the wrong place! It has in CPUs, but they moved to GPUs.
There’s no reason yet to think CPU designs are becoming obsolete. SISD (Single Instruction, Single Data) is the CPU core model and it’s easier to program and does lots of things that you don’t want to use SIMD for. SISD is good for heterogenous workloads, and SIMD is good for homogeneous workloads.
I thought GPGPU was waning these days. That term was used a lot during the period when people were ‘hacking’ GPUs to do general compute when the APIs like OpenGL didn’t offer general programmable computation. Today with CUDA, and compute shades in every major API, it’s a given that GPUs are for general purpose computation, and it’s even becoming an anachronism that the G in GPU stands for graphics. My soft prediction is that GPU might get a new name & acronym soon that doesn’t have “graphics” in it.
I would argue that for many CPU bound games, they could find better ways to utilize the GPU for computation and it is likely they just didn't have the knowledge, time, or budget to do so.
It's easier to write CPU code, every programmer can do it, so it's the most often reached for tool.
Also, at high frame rates, the bottleneck is frequently the CPU due to it not feeding the GPU fast enough, so you lose frames. There is definitely a real world requirement of having a fast enough CPU to properly utilize a high end video card, even if it's just for shoving command buffers and nothing else.
Modern iGPUs are actually quite powerful is my understanding. I think the reason no one does this is that the software model isn’t actually there/standardized/able to work cross vendor since the iGPU and the discrete card are going to be different vendors typically. There’s also little motivation to do this because not everyone has an iGPU which dilutes the economy of scale of using it.
It would be a neat idea to try to run lighter weight things on the iGPU to free up rendering time on the dGPU and make frame rates more consistent, but the incentives aren’t there.
In the high performance scenarios where there is all three (discrete GPU, integrated GPU, and CPU) and we try and use the integrated GPU alongside the CPU, it often causes thermal throttling on the shared die between iGPU and CPU.
This slows the CPU down from executing well, keeping up with state changes, and sending needed data to the keep the discrete GPU utilized. In short, don't warm up the CPU, we want it to stay cool, if that means not doing iGPU stuff, don't do it.
When we have multiple discrete GPU's available (render farm), this on-die thermal bottleneck goes away and there are many render pipelines that are made to handle hundreds, even thousands of simultaneous GPU's working on a shared problem set of diverse tasks, similar to trying to utilize both iGPU and dGPU on the same machine but bigger.
Whether or not to use the iGPU is less about scheduling and more about thermal throttling.
The render pipelines you refer to are all offline non-realtime rendering though for movies/animation/etc right? Somewhat different UX and problem space than realtime gaming.
1. still far from properly utilizing modern graphics APIs as is. Some of the largest studios are close but knowledge is tight lipped in the industry.
2. even when those top studios can/do, they choose to focus more of the budget on higher render resolution over adding more logic or simulation. Makes for superficially better looking games to help sell.
3. and of course there are other expensive factors right now with more attention like Ray traced lighting which can only be optimized so much on current hardware.
I'd really love to see what the AA or maybe even indie market can do with such techniques one day. I don't have much faith that AAA studios will ever prioritize simulation.
You're right that simulation is often an after thought. Most games prioritize narrative, combat, artist's vision and high fidelity environments over complex systems you can interact with. There are a few outliers though.
Alternatively, you get so far into the simulation side of things with stuff like Dwarf Fortress [2] and all visual fidelity is thrown away for the sake of prioritizing simulation complexity!
[0] https://robertsspaceindustries.com/
Seriously, my workflow probably went from spending hours on making something that now takes minutes to get right.
https://jangafx.com/software/embergen/
I was sure that this submission would be about EmberGen and I'm gonna be honest, I'm a bit sad EmberGen never really got traction on HN (https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...)
(Not affiliated with EmberGen/JangaFX, just a happy customer)
I was equally impressed by Spall: https://odin-lang.org/showcase/spall/
https://matthias-research.github.io/pages/tenMinutePhysics/i...
https://matthias-research.github.io/pages/tenMinutePhysics/i...
https://www.youtube.com/watch?v=zxiqA8_CiC4
Their free non-commercial "Apprentice" version is only limited in its rendering and collaboration capabilities. It's pretty... uh... deep though. Coming from software and moving into this industry, the workflow for learning these sorts of tools is totally different. Lots of people say Houdini is more like an IDE than a 3D modelling program, and I agree in many ways. Rather than using the visual tools like in, say, blender, it's almost entirely based on creating a network of nodes, and modifying attributes and parameters. You can do most stuff in Python more cleanly than others like 3ds Max, though it won't compile, so the performance is bad in big sims. Their own C-like language, vex, is competent, and there's even a more granular version of their node system for the finer work with more complex math and such. It's almost entirely a data-oriented workflow on the technical side.
However, if you're a "learn by reading the docs" type, you're going to have to learn to love tutorials rather quickly. It's very, very different from any environment or paradigm I've worked with, and the community at large, while generally friendly, suffers from the curse of expertise big-time.
When I was there we were running ~100 million cell (vehicle) or (vehicle + pad) Hybrid RANS/LES sims with moving overset grids, maybe 10-20 species reactive combustion chemistry (SSMEs and SRBs lighting together), and Lagrangian evaporative particle dynamics (launch water suppression) using Overflow/LARC [5], or Loci/Mississippi State University [6]. Note: My info's a decade out of data, so no idea what the SOTA is these days. More than this.
If you're interested in the field and what a lot of the industry is concerned about, then while old (2014), the CFD Vision 2030 Study is not a bad orientation. [7]
Try to get a ticket to Supercomputing and go wander around. This year was in Denver. [8] Their focus tends to be on "large" though, so you'll mostly get massive weather simulations and stellar cloud dynamics. I like the conference, yet its difficult to get any notice unless you did #CPUs/#GPUs/#FPGAs++.
GOV Outside NASA: NIST (Gaithersburg), DOE (Oak Ridge, Sandia, Los Alamos), Air Force (AF Research Lab), Huntington Beach.
[1] NASA Advanced Supercomputing Division: https://en.wikipedia.org/wiki/NASA_Advanced_Supercomputing_D...
[2] NASA, MSFC: https://www.nasa.gov/wp-content/uploads/2016/01/g-28367g_pst...
[3] Somewhat old examples: https://ntrs.nasa.gov/api/citations/20140016892/downloads/20...
[4] NASA Ames: https://www.nasa.gov/entry-systems-and-technology-division/
[5] OVERFLOW/NASA/LARC: https://overflow.larc.nasa.gov/
[6] Loci/MSU: https://simcenter.msstate.edu/
[7] CFD Vision 2030 Study: https://ntrs.nasa.gov/api/citations/20140003093/downloads/20...
[8] Supercomputing SC23: https://sc23.supercomputing.org/
1. What am I looking for in a job description that would indicate i'd be working on CFD simulations?
2. What would make me a compelling applicant? This seems very different than the distributed systems/recommendation system work that I have a lot of experience in.
Thanks again!
2. A background in distributed systems is actually very good for this field, but high-performance computing (HPC) is a bit different than other kinds of distributed computing, largely because if a node fails or communication breaks down in some way, it is easier to just kill the job. There's no need to make it fail gracefully. Some of your distributed computing knowledge will be extremely valuable but other parts will never come up.
I suggest taking some time and getting familiar with Message Passing Interface (MPI). It is the standard used for distributed memory programming in HPC applications. Almost all of these programs use MPI at least in part. There are several good tutorials online. I recommend [1]. A strong background in MPI is invaluable in a lot of these codes, especially if you can master non-blocking communication.
Learning Fortran may help with some projects and be useless for others. It might be worth the time to at least learn some so that the terminology is not foreign.
I'd also add that many of these jobs are difficult to get even if you have a PhD in the field. Many projects only hire when there is a real need for a new person (or when somebody retires, etc.). Timing the market is an unfortunately reality here. Good luck!
On the NASA perspective, mostly use Linux qsub with MPI. An example of such is:
https://www.nas.nasa.gov/hecc/support/kb/running-multiple-sm...
https://www.cfd-online.com/Jobs/
Many of the positions unfortunately are for grad students and postdocs, but there are a lot of CFD software developer positions posted here. It is a good place to check periodically to see what is available.
Good luck!
From looking over the jobs @atrettel linked to. One other point, is there are actually a lot of different fields other than the "norm". There tends to be a stereotype that its all rockets, airplanes, and cars, yet there's a bunch of others.
Healthcare CFD (Worcester Polytechnic Institute, Ohio State University), Food CFD (University of Michigan-Dearborn), Ocean CFD (Louisiana State University), Plasma CFD (University of Texas at Austin), Civil CFD (UC Berkeley), Petroleum CFD (Pennsylvania State University), Environmental/Pollution CFD (New Jersey Institute of Technology), Weather/Hurricane CFD (Florida International University), Mining/USGS CFD (Colorado School of Mines), CUDA/GPU CFD (Altech LLC), and a bunch of others.
So being from a certain background should not be a worry. There's a lot of industries to lateral in from. The only issue is the limited job pool. However, I started at NASA simply writing software (Perl, FORTRAN, Linux), with a background and knowledge of CFD.
Also, there are definitely jobs (12/24/2023):
Glassdoor, 780: https://www.glassdoor.com/Job/united-states-cfd-engineer-job...
LinkedIn, 50000: https://www.linkedin.com/jobs/computational-fluid-dynamics-j...
Indeed, 800: https://www.indeed.com/q-Computational-Fluid-Dynamics-jobs.h...
It became a book, now in 2ed: Fluid Simulation for Computer Graphics
For a key scene like the balrog, you will probably never decide for simulation and go for full control on every frame instead.
To stay in the tolkien fantasy example, a landscape scene with a nice river, doing a lot of bends, some rocks inside, the occasional happy fish jumping out of water - that would suit a simulation much better.
But I agree with you. You'd ideally reserve key moments for full control and leave simulation for the background polish (assuming your work isn't bound by a physically realistic world).
But real time simulations often use massive simplifications. They aim to look real, not to match exact solutions of the Navier-Stokes equations.
Both use the same technique: Smoothed Particle Hydrodynamics.
But the PS3 was able to fill the whole screen with these particles. Hundreds of them. The game Fluidity seems to have approx. 20.
It had more than 20, but... not by much.
btw, almost unrelatedly, you have no idea how much i appreciate your exegesis of the voxelspace algorithm
Tried them out in Chrome and they're mostly all the same though I do notice a slight jitter to the rendering in the smoke example.