I think that the overall phone size is rather constant, but the bezel on the side of the screen are constantly shrinking, allowing larger screens without transforming the phone into a tablet (yet).
Nexus 4: 133.9 x 68.7 x 9.1 mm
Nexus 5: 137.84 x 69.17 x 8.59 mm
That said, I still find it frustrating -- the Nexus 4 screen is already too large for me to comfortably reach all parts with one hand.