That's what I thought until my first summer internship at a Rails shop. Our unit test suite could take a very long time to run (on my Macbook Air dev machine, it would take 15-20 minutes to run). The slowdown came from testing ActiveRecord models with many dependencies, and that we were creating sample models. As a result, our tests hit the test database multiple times, which made them slow down.
Our solution was to start mocking as many models as we could, which really did improve test runtime.
(Slight edit: after reading the other replies, now I really understand what integration tests are for.)