Metric is already used universally in science and engineering, which seem to be the main fields in which consistency across countries matters. I completely agree that in physics, chemistry, engineering, medicine, etc., it is important to use standard units. But what harm is it doing in practice if in everyday life, Americans say things like "I weigh 180 lbs" instead of 81 kg?
The argument that the metric system makes unit conversions easier (e.g., the fact that it's immediately obvious how many meters are in 20 km, but not how many feet are in 12 miles) is true, but not very compelling to me. I virtually never find myself wanting to do these conversions in a non-scientific or non-engineering context, and on the very rare occasions where I need to, I can always look them up on Google.
A good comparison is degrees Celsius, which are just as arbitrary/unscientific (the standard metric unit is the Kelvin). Somehow people around the world get along fine using Celsius.