The tax code, in all its thousands of pages, is designed (or at least it's supposed to be) to give incentive to certain behaviors that we'd like our businesses (and citizens) to follow. And the more they do this, the less taxes they'll be paying.
Thus, to a first approximation, paying very low taxes ought to be indicative of a payer who's doing all the things that our politicians and bureaucrats think they should be doing.
A few years ago I replaced all the windows in my house, along with a door and a garage door, with new, energy-efficient ones. This entitled me to a tax credit for that year, allowing me to pay a few thousand dollars less in taxes. I don't think that people should accuse me of being a bad citizen because my tax bill that year was unusually low. If anything, I'd think that the same environmental groups who lobbied for such measures in the tax code would all be shaking my hand for taking advantage of that in order to achieve the goal of lowered energy usage.
So yeah, maybe be suspicious of such businesses, but let's not just assume they're evil. That said, the idea that the public has a right to know what corporations are doing this is fully compatible with my claim, so we'd also know who's being a good corporate citizen.