I think it's because the people who want the smaller phone don't just want the smaller phone to the exclusion of everything else.
Some want a flagship model, some want a mid range, some entry level... all in small form factors and they won't compromise in order to settle for the one small model that's available.