Yes, you're right. But a whole bunch of people don't need or use things like automatic refactoring, and more often than not write better code than the people who do.
Presumably, you've got a suite of unit tests etc that you are using to verify your code still works post-refactoring. The same test suite would of course work outside of an IDE environment, so I don't think correctness is a valid argument here (but I agree that speed of refactoring is).
You still review to make sure it was an improvement, and to make sure that the person making the changes didn't make the mistake. But you don't review the correctness of the transformation itself, at least not any more than you need to review the original code.