That’s the idea for next steps.
Basically if you generate a backdrop and then estimate light direction you can inverse render that onto the 3d model given all the depth information you get for free from the model
Basically if you generate a backdrop and then estimate light direction you can inverse render that onto the 3d model given all the depth information you get for free from the model
Though I'm not aware of anyone already doing this, since I think research has moved on to NeRF models that act on 3D scenes directly.