If your unit tests aren't fast enough to run the relevant set after each change, you're most likely writing poor unit tests. And if you need to re-run the whole test suite after making a localised change that doesn't alter behaviour, then either your code is too tightly coupled, or your tests aren't really unit tests, they're integration tests. Maybe both.
I also have to wonder at the sort of attitude that prefers to have faith in Visual Studio or Eclipse, rather than in tests that specify the code's behaviour. And how do you know you didn't mess up using the refactoring tool? Faith in your IDE's correct operation is no substitute whatsoever for checking your code's correct operation.
This whole argument is ludicrous, anyway. A text editor that's been enhanced with powerful plugins is, as far as I can see, an IDE. My Vim setup has project navigation, method completion, build integration, you name it. I could even have automated refactoring if I felt the need:
https://github.com/ecomba/vim-ruby-refactoring
In the end we all use editors with high levels of language-awareness, and varying levels of project- and build-awareness. Some of us put them together ourselves, some of us installed a complete package. Judaean People's Front, anyone?