> One of our biggest problems was that since we tried to change everything at once, the metrics went haywire

Isn't that the problem with empirical methods in general? They are great for measuring the effect of small simple things (like small changes to a web page), but are much less effective in measuring bigger things (like say, programming language A vs. programming language B).