Yeah, that seems to be part of the reason that AlphaZero, which has its System2-like alpha-beta search seems so much smarter than AlphaStar which plays like a brilliant somnambulist.
Maybe if we had some way of abstracting out the things a machine learning system implicitly learns so we could deal with them in a more classical AI-like way?