It seems I'd gotten the wrong impression of how it works. Do you have any recommendations for primers on GPT and similar systems? Most content seems to be either surface level or technical and opaque.
The best source would be the GPT-3 paper itself: https://paperswithcode.com/method/gpt-3