Getting to a solid level of competency in a language actually is not that hard. It just requires dedication and consistent practice.
That said, native speakers operate on a different level entirely. In my recent visit to Barcelona, I had no problems asking for directions, talking to the hotel staff and people in bars, chatting it up with friends in Spanish, etc. However, walking into a noisy / crowded restaurant where the language is going 100 miles an hour is a totally different scenario.
As much as I want to believe otherwise, I'm of the opinion that getting to a level of "automatic" response where you can fluidly handle any situation requires living in a country where the language is in your face all day every day--and even then it still takes real effort.
A friend of mine, who's a US ex-pat living in Madrid is of the same opinion. After three years living there, he still doesn't feel 100% confident; although, he admits that he has spent most of his social time around English speakers.
None of this is intended to discourage anyone at all. In fact, learning Spanish ranks as one of the best decisions I've ever made; however, I think it's important for folks to go in with realistic expectations, as there's big difference between functional competency--with a language and fluency with fluency being a "white whale".