Right now, we have tests, but I'd still say we rely pretty heavily on hoptoad to let us know if we really broke something badly. ;)
https://github.com/thoughtworks/cruisecontrol.rb https://github.com/aentos/ccrb_cap_deployer
The code is simple to understand, self contained, based around rake, and worked almost immediately for us (after some configuration). We were a two person team. I don't know if the setup we had would have worked with a larger team.
It felt like it made an impact in our ability to turn code faster. We had a solid test suite, which helped.