"Maintaining an unsullied self-image" is largely what personal moral codes are about. If I actively killed someone, I would find being a murderer harder to cope with than being a witness to the death of a larger number of people.

I think this is a purely moral standpoint, whereas "killing one to save the five" is the result of people playing a "numbers game." I think this is demonstrated by the opposing results of the "push the fat man off the bridge" dilemma - once you change the mechanism from a lever/switch to actually pushing a dude off a bridge, people's sense of morality comes rushing back.