The original MRL paper (https://arxiv.org/pdf/2205.13147) reported an SVD baseline, which showed comparable performance (Table 1, top-1 accuracy) at d>=256, but much degraded performance at lower dims (d \in {8,32,64}). (Though Table 2, nearest-neighbor accuracy, doesn't show degradation until d <= 16.)

In your conclusion, you report that PCA won on most dimensions. Did you investigate why you found that PCA outperforms MRL when the original paper found that their SVD baseline did not?