Check this post from Joel Spolsky http://www.joelonsoftware.com/articles/BetaTest.html

It's more focused on desktop apps, but it has good insights