Yes, but this is your use case.
While there are a lot of people who share your use case, there are probably a lot more people who don't.
Which is why the product diversity on the market right now is good. There's a decent chance that you'll find a phone that will work for your particular use case.
If the rumors are true and Apple decides to make the iPhone 6 4.7" and 5.5", what do I get next time I'm ready for an upgrade? I think it's silly for Apple to leave money on the table by only selling the current form factor when there's obviously a lot of demand for a larger iPhone, but it's equally silly to ignore the market for smaller phones like every other smartphone manufacturer is.