Look into Montecarlo tree search, CFRM, AB pruning, and also more recent deep learning methods.
It's a very exciting area of research
I think a lot about some advice I heard from the creator of Brogue. Essentially players have a tough time figuring out how AIs make their decisions and often assign complex motives to them when they don't exist. His example was he coded archers to try to maintain a position in some range band from the player as their primary motivation in a vacuum. The community would assign all sorts of supposed logic to archer behavior because in real-world environments with extra geometry or extra situational AI routines they couldn't see behind the curtain. Additionally, that most game AI exists to create a fun experience for the player, we only make it try to play well in service of that goal.