I would assume, more in a fashion close to LSTMs with a hidden state than by re-ingesting all the tokens every time.