Boiling it all down, when prompted, these models just regurgitate a similar sentence to what is observed in the training data for loosely that same input, using some glorified curve fitting. This does not necessarily imply the model understands the meaning of what it is spitting out. So the uninitiated will be really impressed with this kind of toy.
The researchers here appear to have placed particular emphasis on cleaning up what the model is spitting out, but I think it's lipstick on a pig. The area begging for more research is parsing out the meaning of anything but the most simple sentence.