But the moment any of your apps introduce dependencies that cannot be met with your distribution, you are in trouble: maybe the gem needs a newer version of ruby?
Maybe the gem depends on a newer version of a gem that came with the system? Do you manually update that system gem? In that case, you might have just broken a distribution application depending on that older version (and not listing the version it needs, so the latest will be used.
As more and more time passes, the old, safe gem set becomes more and more restricting, increasing the temptation to use gem install instead of apt-get, but once you went down that path, you will open a whole shipping container of worms.
Remember, server distros update rarely. Up until last week, the latest Ubuntu LTS version you could get was from 2010. So much has happened in the ruby world sinc then that sticking to gems of that time could've really hard.
By using rvm, you decouple the gems and ruby your applications need from what the system needs, ensuring that te system remains stable, while you still can use the latest versions of the gems.
Rails itself is normally installed via Rubygems (where it's updated directly by the Rails team), but the problem in this case lies with Ruby: for an up-to-date development/production environment you have to compile it by yourself or use something like RVM.