There is some long term planning going on, but bad luck when sampling the next token can take the process out of rails, so it's not just an implementation detail.