It's done so that the bits will compare the same way whether treated as float or int. (modulo NaNs and stuff)
That is only true for non-negative floating point numbers. It's still useful though.
Interestingly, posits as originally proposed do have this property (except for infinity).
Would have been nice if IEEE stored the complement sign.
It's also practical for hw implementation and has other nice qualities. E.g. comparisons and sorting are easy. Radix sort works for floating point (with some bit magic). Terms and conditions may apply (oddities w.r.t. INF, Nan, etc).
It doesn't seem odd to me. When describing things in nature, (+x, -x) tend to have more symmetry than (x, 1/x).