We can't because we do not care enough. You don't have to boycott Apple largest apple shareholders are pension funds so intermediaries owning shares of Apple on your behalf.
There's a difference between "me" and "we". "We", the majority, could, if we cared. "I", alone or even in a sizeable minority group, can't make a difference, regardless of how much I care.
People can certainly be very self centered, but a big part of this _collective_ culture of either caring or not caring for various things has been manufactured entirely artificially by governments and corporations alike.