Actually I am not very familiar with the internals, I am mostly repeating what Yann Lecun said in an interview a few months ago about autoregressive models.

https://www.youtube.com/watch?v=1lHFUR-yD6I