There are plenty of people in the Ruby community who avoid Rails (whether their reasons are valid or not). I think you'll find plenty of Rubyists who wish to create things outside of Rails. Whether that means they will be throwing weight behind another framework such as Ramaze, or creating a new one, I can't say. I do believe that there are enough anti-mainstream hackers in the mix now that you don't have to worry about that "scene" dying.
You're one of them, to an extent, and there's nothing wrong with that. Have at it.