I was absolutely blown away by how much work it required to get started. That, and how painful it was to explain to a non technical person how to make visual changes.
I was told by a few friends I vented to that what I was doing was "too small" of a project, or "Rails is meant for web apps" which confused me because I assumed a site where you could upload images via a form and then browse and view them was right up its alley.
Edit: A lot of comments are talking about wrapping my head around the framework, MVC, ect. I had no problem building the initial application. My sore points are more about deployments and updating smaller things, like the image of a link. non-technical people don't often remember "oh, and if you want your change to show up, go to this folder, open 'restart.txt' and change it in some way. doesn't matter what, just add a space to the end".