> It seems that float32 is basically just PCM
float has higher accuracy around 1.0 than around 2*24. This makes it quite a bit different from PCM which is fully linear. Which is probably why floating point PCM keeps it's samples primarily between -1.0 and +1.0.
> which was a lossy audio compression
It's not lossy. Your bit depth simply defines the noise floor which is the smallest difference in volume you can represent. This may result in loss of information but at even 16 bits only the most sensitive of ears could even pretend to notice.
> If you give all 31 bits to the exponent, then normalize it to +/-2^7, you get a continuous version of the same function.
You'll extend the range but loose all the precision. This is probably the opposite of what any IEE754 user actually wants.