GPT-3 does only thing, predict the next character based on the previous n characters.

The nuance of the context (previous n characters) and the quality of the prediction(s) (following character, a function of gpt-3 you can repeat by just concatenating its last prediction to the previous n characters) is what has significantly improved.

It has now learned on a significant amount of the writing available on the internet. Which includes guides on writing code, stack overflow, and github repos.

So one is able to write something that looks like a python comment

// This function adds two numbers

And the following n character predictions of gpt-3 will with some unreliability spit out the python code for that function as its predictions

def add(x, y): return x+y

This has been shown to work with simple descriptions of pages, with it returning html with css embedded that matches that description.