The space here is the space of probability distributions (each point is a distribution), not the sample space. So, no real connection with PCA.