When it does it step by step it can look at every step when it writes down its final answer after the steps. There is no extra magic that happens just because you write something in between.

Edit: To expand on that, a humans inner thought would have caught the mistake at that point and corrected it by writing more instead of ending the text. The LLM basically never does that unless you tell it to fix the answer.