And, finally, then it doesn't matter whether your host OS is windows or ubuntu; use which you feel most comfortable with.
Ruby is a bit more understandable, if you don't get DevKit (or if DevKit can't compile something) working with gems can be frustrating. And setting up Git on Windows with an SSH key has been awful in the past.
RailsInstaller takes a lot of the pain out of setup, though.
Another option would be to use something like Vagrant, which gives you a VirtualBox server that's mapped to your local filesystem. So you use the editors you're used to but you have a fairly solid server running the code. If you search around for "vagrant rails" you'll find several pre-built images, along with a RailsCast that explains how it all fits together.
When I get more experience in Rails though, I'll probably switch over to using a Mac.