Another problem worth mentioning is that computing e.g. Euclidean distance you end up adding up things of completely different types, and their relative weights are often entirely arbitrary, e.g. adding distance and weight.

Imagine three points on two-dimension plane: A(1,1) B(5,10), C(2,100). Clustered like this, A and B will end up in one cluster, while C is in another. Now let's scale down the second axis by a 100, now we have A(1,0.01) B(5,0.1), C(2,1) so A and C are clustered together, while B is the second cluster. And that's just liner scaling, imagine how much havoc I could wreck with a logarithmic scale?

I haven't done anything to research this problem, it's just something I came across that made me uneasy about clustering as a whole.