That's a combination of current context limitations and a lack of quality tooling and prompting.

A well-designed agent can absolutely roll back code if given proper context and access to tooling such as git. Even flushing context/message history becomes viable for agents if the functionality is exposed to them.