Edit: I am incorrect. It seems that this uploaded gem was used to compromise RubyGems.org.
> Someone posted http://rubygems.org 's config/*.yml to pastie. Didn't include the S3 bucket secret key, but going to reset everything anyway.
Running `$ bundle update` will not inject this into your app.
You'd have to intentionally add `gem "exploit", "~> 22.31.31"` to your Gemfile.
A more hidden attempt would be to simply alter the existing .gem files on S3. Once they have the access tokens from the configuration YAML files, they can do that from anywhere, not just the rubygems.org host. That keeps their activity under the radar and is slightly harder to track.
In that case, only newly installed gems in your system that were uploaded to rubygems.org before this incident would be at risk. If you run bundle update and pull in gems built from today onward, there is not as much to worry about. If you're updating gems and the bundle is pulling in gems from a few days ago, then you are at high risk.
And, if like most people you don't verify your installed gems (not that you really can if they're not signed), you're going to have a very bad day. The "tracking" aspect is a situation where it doesn't matter because at that point, you're hosed anyway.
Also, walking S3 is not particularly fast. So if the plan is to walk all gem files to verify them, expect that to take a while. Hopefully they have bucket logging enabled.