I can understand that two relatively rotated embeddings from the same or similar dataset can be realigned as long as they don't have internal geometric symmetries. The same way we can re-align two globes -- look for matching shapes, continents.

EDIT: Perfect symmetries, for example, feature-less spheres, or the analogues of platonic solids would break this. If the embedded space has no geometric symmetries you would be in business.

Re-aligning, essentially would be akin to solving a graph-isomorphism problem.

Lie algebraic formulation would make it less generic than an arbitrary graph-isomorphism problem. Essentially reduce it to a high dimensional procrustes problem. Generic graph isomorphism can be quite a challenge.

https://en.m.wikipedia.org/wiki/Procrustes_analysis

EDIT: Sinkhorn balancing over a set of points (say a d-dimensional tetrahedron, essentially a simplex) furthest from each other might be a good first cut to try. You might have already done so, I haven't read your paper yet.