You are not actually killing that person, or destroying that object; you are just deciding which one(s) of the photos of that moment or mood capture it best, and delete the ones which are just slightly worse variations of those.
It takes time though, that is a real cost. Though I think that time is made up for down the road, when you don't have to wade past the photos you deleted, or wait for them to upload/download/backup, etc... even a little bit of self-discipline adds up to quite a bit over a lifetime of digital photography. I think of it as removing weed so the flowers are more enjoyable, and have a greater probability of actually being seen (photographs are kinda useless when they are not seen other than by the bulge they create in the pagination; flickr is like a graveyard where good photos go to get overlooked in that way).
I've been doing this for a about 5-6 years now, so about 3 years ago I started deleting the raw shots a few months after having finished the film. To date I have not once regretted deleting a photo.