It seems to me that the examples in the prompt supply context that affects the conditional probability distribution for what gets generated. These are going to activate parts of the network that give better answers. It doesn't seem any more mysterious to me than the fact that you can ask the network "who wrote this?" and it is also able to sense the conditional probability distribution of the language in the sample.