Why would they care about arbitrary transformations mapping points in the space to other points in the space?
Why would they care about arbitrary transformations mapping points in the space to other points in the space?
E.g. if you have a bunch of particles with fixed mass. You could look at the distribution of speeds and get one entropy. Then the distribution of kinetic energy (basically speed squared). Uniform speed means non-uniform speed squared so the entropies would disagree.
This sounds like it could pose issues.
As for the entropy of probability distributions in general, I think relative entropy is invariant under reparametrizations because both the probability of interest and the reference probability transform in the same way [1]. But I don't remember what does it mean exactly. [And I am not sure if that makes ogogmad wrong, I may not have understood well his comment.]
([Edit: forget this aside. You probably were talking about speeds as positive magnitudes.] By the way using an example analogue to yours discrete entropy wouldn't be invariant either: if you have a distribution {-1,1} and square it it collapses to a zero-entropy singleton {1}.)
[1] https://en.wikipedia.org/wiki/Kullback–Leibler_divergence#Pr...
Actually, I find it hard to come up with a bijective mapping that leads to a non uniform distribution that's useful for anything practical.
Uniformity of v would mean that p(0 <= v < 1) = 1 / 10
Uniformity of E would mean that p(0 <= E < 1) = 1 / 100
But by construction p(0 <= v < 1) = p(0 <= E < 1). So it's not possible for both to be uniform.
But this does bring up a good point. H(X||Y) = H(f(X)||f(Y)) for any bijective f if the distributions are discrete. When they are continuous this is not true, even with a bijective f. For example f = x^2 doesn't work even though it yields a binary distribution. Interestingly however, affine transformations work.