Research on the backspace token: https://arxiv.org/abs/2306.05426
> [...] The IL framework also allows us to incorporate backtracking by introducing a backspace action into the generation process [...]
Very interesting paper, even not considering the backspace stuff, thanks for the link. Pretty cool how that seems to tie in with more recent work on applying pure RL to LLM training.