In that case we shipped literal dirt around the world so that he could put it in his garden. Is that any less absurd?
Just because there is even more out there to miss out on doesn’t make missing out on bananas less of an issue.
I assume you can just stop to eat altogether, because you’re not consuming the vast majority of food out there.
I don't have to try "the vast majority" of other stuff to enjoy the banana.