Small phones were popular when all they could do was make calls and even texting was sort of cumbersome and rare. Of course you'd want that sort of single use device to be as small as possible.
I fail to see how YouTube, Fornite, and Instagram users (to name a few) will want a smaller screen.
I also think the market has settled on rather standard phone body sizes that have not changed significantly since 2014 or so.
I can't comment on what those users want; I don't use any of those things on my phone. I'm not claiming that large phones will cease to exist; I'm guessing that there will be a new trend promoting smaller phones. Some folks will jump on the bandwagon, some will stick with their phablets. Apple in particular will probably make a big marketing push that tries to portray small phones as some kind of cutting-edge innovation.
As another example, Nintendo completely discontinued the non-XL 3DS models in the USA.
I think you aren’t wrong, there are enough people somewhere to buy a small screen smartphone, but apparently that number is small enough to not even open up a single assembly line for.
But those who like content consumption (probably most people) will still be walking around with a rectangle in 10 years, if I had to guess.
Just my own speculation.