Does it know how much output it has already provided when deciding on the next token? How does it decide to start a new sentence or paragraph, or if it's 'satisfied' with its current response?
So to answer your question about knowing how much text it has output so far, not exactly. It is not 'stateful' exactly, it's simply conditioned on previous text and gives a new probability distribution over its vocabulary for the next token. Whether the 'previous text' is its own output, or provided by a human user (a 'prompt'), it isn't necessarily aware of this by default (unless the user vs. system responses are delineated somehow with special tokens for example, which with ChatGPT they may be/are likely to be. My point is that these models don't come with this awareness built in, you have to add it. Fundamentally they simply condition their probability distribution on a piece of existing text).