I have a foldable shopping bag which is smaller than my wallet, you don’t need a full backpack. If I think I might want to pop into the shops while out, I’ll chuck it in my pocket.
Sure, if you want to do a big family shop then it’s a bit different, but that’s more of an event anyway.