I was a bit concerned when I got my wife her Nexus 4 that she would find it too large, but she hasn't said a thing about the size.
Often this is sort of a reasonable plan in general... faster CPUs and more memory are usually helpful (if you can keep battery usage reasonable).
But when applied uncritically and universally, without much judgement or thought, in a mad rush to out-spec the other guy ("Now Double The Size of Our Competitor!!!!"), this just leads to mediocrity, where everything apparently has great specs but nobody really likes them very much...
Same reason iPhone got bigger in a recent major version upgrade. This has been discussed b/f at HN.
I can't think of anything worse than having a tiny screen like the iphone.
Apple prefer to use ratings like JD Power, that group by manufacturer, as that helps those who only target the high-end. Though even on that metric various competitors have gained swiftly and Samsung tablets scored slightly higher than Apple in the latest results.
The nexus 4 had terrible battery life.
Language options being another cavert in your point about small groups. Certainly translate caters for more than 100 languages alone and you can bet this other 10% fit into some of those.
Nothing pleases everybody. Why else do we have exception handerling in programming code :).