If you said "hey I'm bored let's add more squares. Let's move some pieces around or maybe change their movement rules" or "let's play checkers instead" then a human could adapt to this situation pretty easily. A reinforcement learning model like AlphaZero would be like "Whoa whoa whoa not so fast. Let's fire up the GPU cluster." Maybe one day we won't need GPU clusters. But ultimately we're the ones who create all of these tools and decide how to use them.
Why do you treat humans so very different from AlphaZero here? Alphazero could badly play other games quickly, just like humans can. AlphaZero is just able to improve beyond what any human can ever achieve for a lot of types of games.