Speaking as someone who grew up with the metric system before moving to America: Because growing up, we never had to convert between different measuring systems.

Everything was easy to do in your head:

5cm + 15cm = 20cm

0.5kg + 0.15kg = 0.65kg

2900mm / 2 = 1450mm.

Now I'm forced to do mental gymnastics like:

5/8" + 7/16" = ???

2'9 1/4" + 14" = ???

11'4"1/4 / 2 = ???

1lb9oz / 2 = ???

It's not impossible, but it takes me a lot more time to think through these things, and often they are impossible to even punch into a calculator. Adding fractions of an inch in my head was not something I learned in school (honestly, my experience since moving to the US is that most adult Americans also struggle to add fractions of an inch).