This problem specifically (3D reconstruction with representation fitting) is really an overfitting nightmare, they just adapted to it not really overcame it. Nonetheless interesting work.
At that point you're kind of trying to generically train over all possible scene graphs. But maybe some kind of geometric delocalization scheme lets you train on small fragments of larger scenes, transfer learning between scenes, and quickly represent and render generic scenes.