Truth be told, even though I switched from iOS to Android recently (and been quite happy), I have admired Apple when they stuck to their guns w.r.t screen size. It showed to me that they were being sensible about respecting the dimensions of our hands. The iPhone (hardware) just feels quite comfortable to use and hold. Also, iPhone 4's design was excellent - I kinda got used to the distinctive feel of an Apple phone in my hand (with its cuboid geometry).
iPhone 6's exterior just looks a bit strange (now we are going back to the original iPhone?) - it feels off in a way that I can't fully explain. But who knows, maybe it would be just fantastic to hold and use. [Like how the quick reports coming out from people who have held the Watch saying that it feel really nice]