Buffett/Berkshire is literally the only example I can think of where they only buyback shares when prices drop below a certain threshold. Most companies do it regardless of whether it is creating or destroying value, based only on whether they have cash authorized to spend.
If that is a shareholder's opinion, they should sell the stock.
Buybacks tend to happen at the worst times (boom years) and cut out in recessions which is counter to what you'd want as an investor. Though if you think of it more as an alternative to special dividends it makes more sense.
It doesn't make sense for shareholders to own stock that they think will go down. If they really think that, they should sell. So shareholders should think the stock is either fairly valued or undervalued, almost by definition.
Any shareholders who don't sell during a buyback are compensated by owning a larger share of the company at what should be considered a fair or generous price from their point of view.
The place where this breaks down is when you believe a company would have a higher intrinsic value, provided that it has sufficient financing. But this isn't based on the stock price; it's based on your theory of how much money you think the company will need.
Also, from a non-shareholder's point of view (say, bondholders), a buyback means the company's ownership changed, it has less cash, and they didn't get anything for it.
Do you have evidence of this? Shareholders vote on what to do with the corporation's money. I would expect the vast majority of those decisions to be rational.