You don't miss it in practice because US society isn't built around it in practice.

There's something great about being able to print something at home on A4, see how it looks, and know that it's going to be exactly the same (no stretching or cutting required) when you go to a print shop for an A1 or A0 print.

And yes there is a lot of legacy network effects around Letter size, which is why the switching costs are very high to make switching impractical.