While not opting out of most of the convenience the Apple ecosystem gives me, I did buy a GRIII and carry it along everywhere I go - together with an otherwise capable iPhone. The difference in optics and sensor really does show in print or when cropping.
I have one of these and yes, I can cram it into my pocket with some effort, but it isn't very testicle friendly.
Your trousers? No. Your coat? Definitely. Unless you go for full-format ones with bigger lenses. In which case your priorities most likely aren't carrying your camera gear in your pocket.
I’m not wearing a coat all year long, and almost never on vacation. The latter situation is the most important one where I need a less bulky solution.
Fuji X-E4 + 27mm f/2.8 at 2" thick is quite pocketable.