This is so true. While I haven't given many speeches in my life yet, I've had some experience playing in public at concerts (max I had was 500 people probably so not a huge crowd, but still plenty).
Every time I would step on stage with my band, I'd feel like I locked up, unable to do anything other than just stare at the crowd. It's not really anxiety in my case, more like a "oh crap, what if I fuck up?". Thanks to practice with the other members and eventually having fun (once the music starts), you tend to forget it all and just get into action. The same happened during my graduation speech, once you start the only option left is to get to the end.
More often than not I got to the end of a concert thinking "that was all?" because time really flies when you put all of yourself into doing a very focused thing (and speeches aren't that different, at least in my experience).
Also, the best thing you can do to recover from stumbling is to just keep going. If you realize you have screwed up something, just keep going and everything will be fine. The thing you definitely don't want to do is to lock up and freeze. The moment you freeze you know you have failed. The worst experience to me was during an acoustic solo part when a string in my guitar broke, I was just there, standing in front of a crowd and then silence. We just laughed it off and kept playing (with a substitute guitar), but it felt terrible for those first couple of seconds.
ps: shameless plug if somebody cares, here's my band: http://www.youtube.com/user/HavenlostBand