Basically why I bought a Moto G⁷ Power: it's about the same size as all the other phones these days, and it actually has two to three days of battery life. It's about the thickness of an iPhone 5, which was fine.
I personally think the iPhone 4S's display was a good size and resolution, and the form factor is good for human hands. If they could close in the bezels a bit, cut off a tiny bit of chin and forehead, and make the shell like the iPhone 5C... and make it run Android... I'd probably be game.
They could probably procure a really nice display in terms of colour, switching speed, sustained brightness, and black levels, maybe even increase the refresh rate to 120Hz for the interaction latency benefit. At 960x640, the quality of each of those pixels could be spectacular, and it'd further help with the end-to-end latency.
For the last couple of years I didn't ever bring a smartphone with me, just my trusty Nokia 106, which itself gets about a month of standby time per charge. I still carry the 106 because I'm still not used to having to charge a device so often.