You should feel bad if you enforce it.
Practically speaking, if I hadn't had a patent granted while I was doing my first startup, I would've never gotten the acquisition valuation I got. It's a part of the game, you just have to have an "IP." Do I think this is retarded? Sure. Do I feel bad that I got a patent? No.
A patent making its way to a patent troll is certainly not a good thing, but being an unlikely event it needs to weighed against the benefits of having a patent.
With regards to the buyers enforcing the patent, this is easily handled by obtaining their commitment to use my patents only in a defensive capacity. In my case I didn't have to get an explicit commitment, because I knew the company and its internal ethics well, but in a more general case it's really not a show stopper.
Context that matters the most. Groups can either try to reclaim the word or start using new ones, which I believe many are doing.