This is cool also BabylonJS has nice gaussian splat support as well: https://doc.babylonjs.com/features/featuresDeepDive/mesh/gau...
How do Babylon, Aframe, Three.js, and PlayCanvas [2] compare from those that have used them?
IIUC, PlayCanvas is the most mature, featureful, and performant, but it's commercial. Babylon is the featureful 3D engine, whereas Three.js is fairly raw. Though it has some nice stuff for animation, textures, etc., you're really building your own kit.
Any good experiences (or bad) with any of these?
OP, your demo is rock solid! What's the pitch for Aframe?
How do you see the "gaussian splat" future panning out? Will these be useful for more than visualizations and "digital twins" (in the industrial setting)? Will we be editing them and animating them at any point in the near future? Or to rephrase, when (or will) they be useful for the creative and gaming fields?
One of the Spark goals is exploring applications of 3D Gaussian Splatting. I don't have all the answers yet but already compelling use cases quickly developing. e.g photogrammetry / scanning where splats represent high frequency detail in an appealing and relatively compact way as you can see in one of the demos (https://sparkjs.dev/examples/interactivity/index.html). There are great examples of video capture already (https://www.4dv.ai/). Looking forward to seeing new applications as we figure out better compression, streaming, relighting, generative models, LOD...
A-Frame hello world
<html> <head> <script src="https://aframe.io/releases/1.7.1/aframe.min.js"></script> </head> <body> <a-scene> <a-box position="-1 0.5 -3" rotation="0 45 0" color="#4CC3D9"></a-box> </a-scene> </body> </html>
The good:
1. Blender plugin for baked mesh animation export to stream asset is cool
2. the procedural texture tricks combined with displacement maps mean making reasonable looking in game ocean/water possible with some tweaking
3. adding 2D sprite swap out for distant objects is trivial (think Paper Mario style)
The bad:
1. burns gpu vram far faster than normal engines (dynamic paint bloats up fast when duplicating aliases etc. )
2. JS burns CPU cycles, but the wasm support is reasonable for physics/collision
3. all resources are exposed to end users (expect unsophisticated cheaters/cloners)
The ugly:
1. mobile gpu support on 90% of devices is patchwork
2. baked lighting ymmv (we tinted the gpu smoke VFX to cheat volumetric scattering)
3. in browser games essentially combine the worst aspects of browser memory waste, and security sandbox issues (audio sync is always bad in browser games)
Anecdotally, I would only recommend the engine for server hosted transactional games (i.e. cards or board games could be a good fit.)
Otherwise, if people want something that is performant, and doesn't look awful.... Than just use the Unreal engine, and hire someone that mastered efficient shader tricks. =3
In general, most iOS devices are forced to use/link their proprietary JS vm API implementation. While Babylon makes it easier, it often had features NERF'd by both Apple iOS, and Alphabet Android. In the former case it is driven by a business App walled garden, and in the latter it is device design fragmentation.
I like Babylon in many ways too, but we have to acknowledge the limitations in deployment impacting end users. People often end up patching every update Mozilla/Apple/Microsoft pushes.
Thus, difficult to deploy something unaffected by platform specific codecs, media syncing, and interface hook shenanigans.
This coverage issue is trivial to handle in Unity, GoDot, and Unreal.
The App store people always want their cut, and will find convenient excuses to nudge that policy. It is the price of admission on mobile... YMMV =3
Would you have any suggestions on what JS/TS package to use? I built a quick prototype in three.js but I am neither a 3D person nor a web dev, so I would appreciate your advice.
Examples:
- https://audiolabs-erlangen.de/media/pages/resources/MIR/2024...
- https://images.squarespace-cdn.com/content/v1/5ee5aa63c3a410...
1. Use global fixed 16bit 44.1kHz stereo, and raw uncompressed lossless codec (avoids gpu/hardware-codec and sound-card specific quirks)
2. Don't try to sync your audio to the gpu 24fps+ animations ( https://en.wikipedia.org/wiki/Globally_asynchronous_locally_... ). I'd just try to cheat your display by 10Hz polling a non-blocking fifo stream copy. ymmv
3. Try statically allocated fifo* buffers in wasm, and software mixers to a single output stream for local chunk playback ( https://en.wikipedia.org/wiki/Clock_domain_crossing )
* recall fixed rate producer/consumers should lock relative phase when the garbage collector decides to ruin your day, things like software FIR filters are also fine, and a single-thread output pre-mixed stream will eventually buffer though whatever abstraction the local users have setup (i.e. while the GC does its thing... playback sounds continuous.)
Inside a VM we are unfortunately at the mercy of the garbage collector, and any assumptions JIT compiled languages make. Yet wasm should be able to push io transfers fast enough for software mixers on modern cpus.
Best of luck =3
PlayCanvas Engine: https://github.com/playcanvas/engine
PlayCanvas Web Components: https://github.com/playcanvas/web-components
PlayCanvas React: https://github.com/playcanvas/react