I can fit the larger phones (though still prefer the SE size), but my wife has demonstrated how even the SE is often spilling out of whatever pocket or other way-smaller device holding pocket is on her clothing items. Unless fashion trends change, or everyone who uses purses goes for some giant bag (and likes keeping their phones in a bag, not on their person), there is a large number of people who could use a smaller phone.
A good pocket in women's clothing is unicorn level fashion.
I however always have a bra strap. I end up storing my phone there most of the time.
I know a lot of women who find the pocket situation in women's clothes very frustrating. They feel like they're being dictated to that showing off the line of their hip is more important than actually being able to carry anything useful.
Buy this $1000 phone, and you can always have it along if you replace your entire wardrobe!
The GameBoy solution: Just have a tailor make pants with pockets that fit the device.