> NeRF needs something equivalent that is physically grounded.

Have you seen Plenoxels? https://alexyu.net/plenoxels/