If oysters - lacking a CNS - react to external stimuli and quite probably "feel" something, even something as basic as lack of food, shouldn't we abandon the idea to live without hurting some other living being?

Even in the most ideal circumstances we kill other beings simply due to resource consumption. Maybe not now, but in the future - what we consume isn't available to them. Even if you claim the resources we consume aren't food to the food species of your choice, due to the law of increasing entropy we definitely shorten the lifetime of whatever comes after us just by existing.

Hardcore buddhists for example consider all life equally worthy. No karma bonus for vegans, maybe less than for non-vegetarians who buy only from farms which provide a healthy, livable life to the livestock (or, obviously, for the plants)