As others have said, it's the middle-endianness itself which is weird. Also, if you're going to write the whole date anyhow, I don't think it makes a huge amount of sense to describe any part of it as "more important" to the extent that it deserves re-ordering a date.

The evolution of the US ordering feels (I might be wrong!) similar to how spoken numbers are little endian from 11 to 19, yet big endian from 21 to 99. Some languages are even worse in this regard; but regardless, they're unhelpful historical artifacts that we learn to live with, not actually helpful, well-thought-out, or even merely irrelevant. Notably there are considerable differences between how quickly children learn maths in different languages, and digit ordering may contribute (of course other cultural factors may contribute too). I wouldn't be surprised if date ordering too is similarly a small but real drag on learning.