As far as I understand from a quick read of the paper, the 2D diffusion doesn't have a 3D understanding. It probably have some sort of local neighborhood understanding, aka small geometric transformation of objects map close to each other in the diffusion space (That's why like with latent spaces you can "interpolate" (https://replicate.com/andreasjansson/stable-diffusion-animat...) in the diffusion space).

But that's not really surprising because when you have enough data, even simple clustering methods group objects like faces by the direction they are looking to. With enough views even a simple L2 distance in pixel space allow t-SNE to do that.

They are injecting the 3D constraints via the NERF and an optimization process to add the consistency between the frames.

It's a deep dream process that optimize by alternating updates for 3D consistency, and updates for text-to-2Dimage correspondence. It's searching for a solution that satisfy these two constraints at the same time.

Even though they only need to run a single diffusion step to get the update direction, this optimization process is quite long : 1h30 (but they are not using things like instant Nerf (or even simple voxel grids) ).

But this will allow for creation of a dataset of 3D objects with corresponding text, which will then allow to train a diffusion model that will have a 3D understanding and will be able to generate 3D objects directly with a single diffusion process.