My theory is that big phones dominate because a lot of people don't have reliable/maintained computers other than their cellphone. If that was true of me, I'd want the biggest cellphone I could get, too.
Personally, I think if this is true, and phone companies figure it out, you'll see phones begin to be priced like turn-of-the-millennium laptops. The giant and clunky "desktop replacement" units should be giant and priced at the bottom of the market. The small units should have all the premium features and be way more expensive.
The fact that there are very few small smartphones being made and that none of those are 'premium' smartphones would indicate that either I am wrong or there's a market opportunity that the major cellphone manufacturers are missing.