This is very fascinating, so thank you for looking into this. I think you may have misunderstood the rating system. You said

>>>Let’s say I really have to produce a rating, though. What would I do? I would probably: - Find some experts and ask them to assign weights to my various measures on the basis of how much they contribute to quality

But isn't this is exactly what the latent variable model (really a PCA) is doing? The only difference is - rather than have experts pick 60 weights for each component, which would require 60 contentious decisions, the PCA does some form of dimensional reduction so the experts need only pick weights for 7 components, which "unravel" into 60 weights. This sounds reasonable to me - assuming of course, the PCA components each have meaningful interpretations, and measure the degree of "good".