Then I got a Galaxy Note 2. The day I got it was literally the last time I used the Nexus 7. It did everything the Nexus 7 could do and it was always with me. It had almost the same display resolution in a much smaller overall package thanks to the slimmer bezel.
It was bigger than the Droid Charge, but it actually fit better in my pants pocket. My carrying system for years has been to put the phone (with no case) in my left front pants pocket with the screen facing in to protect it. Keys and change go in the right front pocket so they never scratch the phone. Don't need cargo pants for this, just regular slacks or shorts.
This worked OK for the Droid Charge, but the phone always ended up sideways in my pocket. When I sat down, the Droid would stick out because it was crosswise with the long end against the curve of my leg. The Note 2 was big enough that it wouldn't tip on its side in the pocket like that; it stayed upright with the long end parallel to my leg and the short end going across. So despite being a bigger phone, it ended up being less in the way.
Later upgraded to a Note 3 and Note 4, and wouldn't go back to a smaller phone at all.
I do pay attention to pocket depth when I buy pants, but most of the styles I like have reasonably deep pockets anyway.
I don't talk on the phone much, and when I do the larger size isn't a problem. I mostly use my phone as a portable always-connected tablet.
For anyone who thinks these large phones exist only to give manufacturers bigger numbers they could brag about, well, I'm your counterexample. They are ideal for me.