But wouldn't (couldn't) the model then hallucinate play patterns and get itself into problems when playing against a real opponent?
When the Dota 2 bot was made, they retrained the bot only partially when new patches came in, so it was definitely cheaper to adapt.