The argument about childhood is specifically about incremental physical transformation being entangled with cognitive development. "switching out [bodies] at any time" is not in any sense equivalent.

Actually that's pretty much exactly the theory I've seen suggested a few times, Harold Klawans is a big believer for a start. He makes a good case in his books that human intelligence is very much the result of the fact that much of our brain growth occurs post-birth as our bodies increase in size, which makes us quite different from most other species. It's not just that our brains absorb knowledge as we grow into adults, but transform themselves radically in order to gain particular skills (particularly language processing).

The folks who believe that a significant part of our cognitive function is rooted in our physical selves certainly believe that. There's even an entire book "Philosophy in the Flesh: The Embodied Mind and its Challenge to Western Thought" by George Lakoff & Mark Johnson that argues that not just cognition but almost all of western Philosophy is grounded in our physical manifestation.

Not a robot.