Hacker News Reader

Comment by streptomycin | Hacker News Reader

Open Matlab and type

    load hald;
    coeff = pca(zscore(ingredients))
    ingredients(:,1) = ingredients(:,1) * 2;
    coeff = pca(zscore(ingredients))

Magically, you get the same result regardless of if you change the units of one variable.

View on HN

Yes, of course you will--because you've normalized the data. If you had run PCA on just ingredients instead of on the normalized ingredients, I would imagine the results would be different. I can't verify since I don't have matlab on this computer, but doing PCA on raw data with one set of units will produce a different PCA doing it on data with another set of units.

Yes, you get different results if you don't normalize. My point was, I don't see why "you have to trivially normalize your data first" is a meaningful argument against anything.

Because normalization reduces the influence of variables that have a higher variance. In raw data, if you have marathon times and heights for runners in a race, and you measure the times in minutes and the heights in inches, the influence on the principal components from the times will likely be much more.

Like all problems in statistics, it ought to depend on the specific task at hand. If there is some a priori reason to use the original scale (or a different re-weighting), it ought to be used. In general, PCA on correlation matrices is much preferred for exactly the reason you mention.

Reply on news.ycombinator.com