Can I put a pod back inside, with SSH, and use it to kubectl exec my way into an interactive state?

I get why the "outside" has no SSH, but I would not preclude the existence of a shell inside, busybox, you're back in the land of the UNIX living and all that entails.