A lot of people also complain about the performance of strings in Haskell, so this is right where the tradeoff is today. Tomorrow it won't likely be an issue.
A really smart compiler should be able to optimize most of the inefficiency away; I'd bank on compilers improving fast enough to make seemingly inefficient things cost much less.
If I'm designing a language today for the future, I'd err on the side of usefulness.