You will always be able to ask the agent to do refactors for you - and it can do mega ones that exhaust you to think about!
The idea of setting up an agentic loop to review code and propose and implement refactorings still seems pretty awful to me, though, yeah. Maybe cut that off at the first green-bar revision, and then apply some actual taste and judgement.