I haven't actually heard anyone complain about it or get annoyed at some group of people - this whole concept of being annoyed at some group of e.g. environmentalists actually seems completely strange and alien.
Yes, it takes a bit of getting used to, but now it's simply a habit to take a small folded up reusable bag most places when we head out (just incase we end up shopping), or to think ahead and grab some bags when we know we're going out for a shop.
Really not a big deal.