I remove labels too when possible, and avoid clothing where the brand is an integral part of the item. Why would I keep them?
However, the vast majority of logos I see on clothes are actually part of the clothing, that's what I meant by "stitched in." I realize now, via Google, I meant "embroidered on."
EDIT: Just went through my closet and I found there's another large category of logos that are screen printed on. There were zero I personally saw that could be removed with a knife without heavily damaging the fabric. Maybe men's clothing is different.
With men's clothes, you can buy plain shirts, socks, etc in 3-packs (there's few choices, but few are needed). Women's clothes tend offer more unbranded options, unless you're buying athletic wear or a purse. Casual shoes generally suck in this regard, for men and women.
This has been getting more difficult, of late. It used to be that logos were embroidered on tags or patches, which were then stitched onto the clothes. But with embroidery robots and the rise of counterfeiting, companies are making it more difficult to remove their brands.
edit: but practically? Thrift shops have a wealth of logo-free stuff
A blank patch looks sort of conspicuous, but a cute little non-corporate owl or fox will look like it's meant to be there.