I mean riding a horse also requires more skill, and so does braking on slippery surfaces without ABS. Personally I'd rather not do the latter, and not rely on the former for transportation...
But to each their own. Maybe I'd me more inclined to drive manual if I wasn't living on a hill... surrounded by hills.