I tend to ask, not how you use the framework, but rather how the framework works. How does routing actually work? How do named scopes work? How would you implement the {before|after} filters on models... etc.

It's actually somewhat more difficult to tell whether someone knows how to use Rails well. My only suggestion is to sit down in front of an editor with them and work through actually building something small but meaningful.