In most strategy games, rule-based AI is easier to author, contextualize, and theme. Players usually prefer AI agents with memorable and varied traits instead of excellent high autonomy (complex utility, HTN, ML) AI. They like evident strategy more than complex strategy. Smarter and harder AI sounds appealing on paper, but it’s often less fun for most players who want a predictable, beatable opponent.
High-fidelity human-level opponent AI is only really appreciated in eSports, where it serves a training purpose. However, eSports people have fun in an aim trainer—they are not the “normal” gamer.
Most people want “really weak chess” and wouldn't enjoy hard or complex chess. They like game AI opponents acting in simple and vulnerable patterns they can outsmart and feel clever. And the more they dominate the agents by learning them, the more they like the game. Even if they say they don't, the user testing shows they do. The cheat bonuses that are super easy to win against are probably intentional :)
There is also another reason for building simple game AI — the illusion of intelligence is not created by complexity but rather by anthropomorphization/personification. It's the same as with LLMs — the big breakthrough was when they started speaking like humans, and all further complications aren't adding much to the broad and intuitive feeling of their intelligence. Similarly, a game AI agent that tells you what they are thinking achieves more perceived intelligence than an AI agent that is actually more intelligent. With this in mind, game developers don't waste much time on deep intelligence. This time is better used making agents bark, animate, display human reactions, show context awareness, etc. It is very much about the agents being showhorses, not workhorses.
I say this from pretty long AAA game AI design and architecting experience. It’s wiser to do what Firaxis did.