I'd highly recommend using `bundle gem #{your_gem_name}` to generate a template for a new gem. This will create sensible directory structures, stub out a gemspec, create a .gitignore, and add some rake tasks for building and pushing to rubygems.org.
I would recommend using newgem from DrNic. It's gem template generator that comes with nice defaults.