No company since at least 2016 has made a
flagship Android phone under 4.5". The iPhone SE made news for being the only small phone with up-to-date internals. It's not surprising small Android phones have seen limited success.
That's like observing that small cars don't do well with luxury buyers. Well, true -- because carmakers have chosen not to make small high-end cars. I've had friends tell me they wanted to buy a smaller car but the interior wasn't nearly as nice so they bought a bigger model. They'd rather have a car that's too big than get a second-rate dashboard and seats.
So I'm not sure why you're pointing to supply and claiming that it's indicative of demand of one particular attribute. Even more than with cars, the history of cell phones is companies making the wrong thing, and then having their lunch eaten by some other company. 15 years ago, all my friends had Blackberrys or Palms, but today nobody would say "The market had spoken" and everybody wanted hardware QWERTY keyboards and styluses. People didn't have a choice. These companies were seeing exponential growth, right up until a better choice was available.
Also, the desktop display comparison is out of left field. My desk is plenty big enough for two big displays, while my pants pockets aren't big enough for even one 'phablet'. I don't have to move my fingers across every pixel of my desktop displays to use them. It's completely different -- ergonomics, use cases, hardware, OS, apps, and so on.
Whenever there are fewer companies C than the exponent 2^R of the number of relevant attributes R, consumers don't have a sufficient number of axes of control to communicate to product designers purely through market forces.
(There's a name for that law, in the context of systems control theory, but I can never remember what it is.)