It could be that the keyboard is actually amazing, that the concerns are overblown, and that the engineers, developers, journalists, and consumers that have been complaining about it are either lying for some reason or lack the perspective to realize what is and isn't important to their workflow.
It could be that it would be a huge public image loss to admit that they were wrong about this. No matter how Apple phrases it, it's always look bad to say "we've spent the last 2-3 years telling you this was a technical achievement, turns out we were wrong." One of Apple's biggest marketing angles is "we put the work in and get it right the first time." That's going to be something that people mock them over, regardless of whether or not it's the right decision to make.
It also might open the floodgates on more expensive litigation and warranty requirements in the future. Apple's current warranty that they just rolled out is only valid up to 3 years after the initial purchase. Is a judge more likely to force them to extend it if a lawyer can argue in court "they lacked so much confidence that they reverted their own design?" Are their significant investments into blocking Right to Repair going to be hampered by that kind of public admission?
They've also invested large amounts of money into the current manufacturing process and design. Their recent decision to discontinue the 2015 model might point to this being about manufacturing costs - you could ask the same question of that decision: "why not allow the holdouts to keep purchasing the older model?"
Along with that, it could also be sunk-cost fallacy at play. One way to check if Apple has a problem with sunk-cost is to look back in the past to see if they've exhibited a pattern of doubling down on controversial decisions and rejecting criticism or blaming customers for issues.
It also could just be that the keyboard looks sexier in advertisements, and perhaps Apple optimizes for advertisements over extended customer experience because they have enough built up goodwill and reputation to do so. The devices might sell better right now when marketed as futuristic status symbols, rather than as practical machines.
Finally, don't dismiss the idea that it could just be the result of designers and engineers running wild without enough practical input to reign them in. I'm all for giving companies the benefit of the doubt, and I understand what you're getting at. But you should apply your philosophy in moderation or else someday you'll find yourself defending Microsoft Bob. Companies are made of people after all.