In technical terms it lets you stop the transformer at any token, sample alternatives from the probability distribution at that point, pick your desired next token and resume the generation.
This UI lets an expert guide and control the output of the tool, instead of having to fight it. I wish there was a frontend for large GPTs that worked like this instead of the ubiquitous-yet-useless chatbot.
The input and output of a GPT it not a sequence of messages in turns: fundamentally the input is the entire context (both from the user and the model), and the output is a probability distribution for 1 single next token.
The chat is only one of many interaction patterns possible with that primitive: one where the "most likely" output token is concatenated to the previous input and this is repeated until a limit is reached or the model produces a special "end of sequence" token.
Representing a more complete interaction pattern as a chat is impossible because the chat abstraction is lossy and could never represent it. Sometimes the abstraction leaks, as with the concept of "temperature", that's not a parameter of the model but of the chat.