I thought you had a point. Professional basketball comes to mind, where it's incredibly unusual for someone to be of "normal" height. A height difference gives you an advantage which I suspect creates a positive feedback loop (advantage leads to more enjoyment, which leads to more practice...etc) and remains a significant factor even when all other things are equal (like innate talent).