I think diminishing return kicks in around 6'2"... plane seat, clothes fit, cramped cafe / restaurants,
I'm pretty short, but I'd much rather be 6'2" than 6'4".
As someone who is 6' 4", I agree. Not that I'm complaining, but given the choice to pick my height, I'd opt for 6' 2". For pretty much all the reasons you mentioned but also because the world is built for people closer to average height. Think kitchen counters, plane/bus seats, stairwell ceiling clearance, etc. I'm relatively young but have back problems that I partially attribute to my height.