Oh, for sure. The key though is that the value of "it" which I am making worse is not a value of "it" which I have any interest in making better. Nor would I ever suggest otherwise.
In your mind I may be nothing but an input into your total utility function, but there's no reason for you to expect me to care about maximizing it.