Designing for the "average person" suits almost nobody. Think ergonomics, car seats need to be adjustable for safety. And people vary mentally a lot more than they do physically.
If you don't design stuff with an average user in mind, is literally impossible to build anything. You would never run out of edge cases to cover.
EG: You design a chair. You design it up to hold up to 250lb, then you start going thinking it further and say "ok there are 10 people who weight 550lb, so yeah lets build all the chairs that way even if they now cost 1200$ each". No, what you do is design a regular chair that fits 80% the people and then if you want you can design one who fit 90%, another with 95%, 99% or how much ever you want. You always need to decide what the cut-off point is, and it's usually mandated by costs.