Of course, this is all theoretical if you can't change what makes you happy—and, since psychological studies say that how much you can change your utility function is something that varies individual-to-individual, there will never be a widely-accepted belief that it's impossible to decide what you like, as there will always be some people who can.
The problem in many people's minds is not with the consumers of porn but with the creators or actors of porn, especially the women. Like a lot of other so-called sex work fields, many human rights advocates end up torn: on the one hand, it's a field where women can make money with a certain amount of independence; on the other hand, the field can be violent, unsafe (drug addiction is rampant, sexually transmitted diseases are a constant fear, etc.) and (arguably) is enormously degrading to the spirit and the bodies of the women it employs.
Anyhow, I think that people who want to clamp down on porn are generally worried about the producers rather than the consumers. (In fairness, one influential argument from Catherine MacKinnon argues that porn makes the men who consume it more likely to be violent towards women. But I think that argument is the exception not the rule.)
Porn will find ways to users, one or the other way anyway, so it's also pointless.