I was saying the same thing a couple of years ago. I held on to my SE as long as possible because I didn't want an iPad phone. "There's no way I'm upgrading," I said. I thought those with iPhone Plus' and Galaxy Notes were either giants or hard of seeing.
However, I have the X now and I can't imagine going back to anything smaller. When I look at my buddy's SE now the screen looks so tiny. I guess the moral of the story is, "Don't knock it until you try it... at least for more than a few days."