Massively Parallel Rendering of Complex Closed-Form Implicit Surfaces (2020)
mattkeeter.com
mattkeeter.com
For example, gaming pixel maps have been used for semantic simulation, and have been rendered scenes: https://arxiv.org/abs/1608.02192 https://www.cv-foundation.org/openaccess/content_cvpr_2016/p...
This concept has not (yet) been applied in audio ML. We have a paper in submission---will be on ArXiv soon---where we share a GPU-enabled modular synthesizer that is 16000x faster than realtime, concurrently released with a 1-billion audio sample corpus that is 100x larger than any audio dataset in the literature. Here's the code: https://github.com/torchsynth/torchsynth
Closed form implicit surfaces are actually one of the most difficult way to model the real world. They are neat because they are very compact and the creation process is close to modeling with (mathematical) clay. But they are hard to use if you want to model the real world with all its complexity, resulting of a variety of chemical and physical processes happening over time. There is a reason why they are so popular in the demoscene, for which technical achievement and art is more important than realism, and not much elsewhere.
The paper is about making rendering of these primitives more efficient, which may prove to be a great addition to an artist toolbox, and maybe for scientific imagery. However, I don't really see applications for ML anytime soon.
If you enjoyed this paper, there's a companion blog post about the actual process of writing it: https://www.mattkeeter.com/projects/siggraph/
(and I'm happy to answer questions, of course)
The downside to sphere tracing and similar is that it limits the input model: you have to guarantee that evaluating the model at [x, y, z] gives you a result that's less-than-or-equal to the true (Euclidean) distance to the shape's surface.
(or a distance adjusted by some constant scaling factor, i.e. Lipshitz continuity [2])
This is a relatively fragile property, and really limits what kinds of shapes and transformations you can use when modeling.
Using interval arithmetic is more robust against arbitrary models, at the cost of being less efficient when models are well-behaved.
I don't know much about state-of-the-art fractal rendering! I'd imagine that the fixed (original) tape in MPR would be a limitation here, because you may want to terminate conditionally, rather than evaluating a fixed expression.
[1] http://www.fulcrum-demo.org/wp-content/uploads/2012/04/Cone_...
https://www.youtube.com/watch?v=_6CnaugAcCc
Highly recommended.
I guess they simply didn't do much work on that part, using some bruteforce-ish raymarching technique, which their fast evaluation and nicely bound objects allows. They mention further work though, like sparse voxel octrees, improving culling, etc... So I guess that will be for a "future episode".
(There's also a bit of extra logic to skip regions which are occluded in Z, plus a final pass to render normals using automatic differentiation)
Same priciple should work for anything with decently good distance function, reifying a mesh or SVO could always be combined with rough culling of the "tapes" since it's all about the distances.
Need neural radiance fields. Then add super resolution, then add motion prediction and you are on your way to a synthetic visual cortex.
In the future GPUs will be spec'd by how far in the future they can predict given a power and thermal envelope.
In another project [1], I found a 2-6x speedup in going from an interpreter to a fully-compiled shader, so this can make a huge difference!
how do people that absolutely need the lowest latency numbers make do with GPUs? i'm not in graphics but i'm in ML and lately i've been working on research to squeeze as much juice out GPUs as possible. my last project involved optimizing a pipeline that basically consisted of just a stack of filters and some min/max finding. the fastest i could get it to go after throwing everything i could at it and i only got it down to ~20ms. that's ~50Hz. admittedly it was a tall stack but still i don't understand how game devs (for example) get complex pipelines to finish within the 60Hz/16ms given you don't never have access to bare metal GPU.