I think for AI the core reason is more sophisticated AI isn't particularly great from a game design POV. In a game, predictability is key, both because a) it allows the game designers to actually change the AI behaviour to balance it and be able to incorporate it into their game design, and b) if the players can't predict the AI they can't plan around it, and the ability to make a plan around a game's systems and try to execute it is a key part of non-frustrating gameplay in most games. The more state-of-the-art AI systems tend to become less predictable and for the most part that makes the games less fun.
In terms of physics usually the issue is if you have a rich physics envrionment it gets more fragile: it's easier for players to break it, intentionally or unintentionally. I've played some games with physics systems far in advance in terms of number of objects than could be achieved previously and they are all pretty much just 'buggy', not necessarily in terms of coding quality, just in terms of how rigid body physics is really prone to creating edge cases which create massive forces, velocities, and/or accelerations from apparently benign arrangements of objects.