There is also a wealth of plugins for buffer switching; I like FuzzyFinder's :FufBuffer command. Invoke it and the most recent buffer will be at the top, so you can hit enter immediately.