The ratio seems pretty good to me. A lot of people just don't want to give out their email address.
1. Many will try again later with a throwaway email account they created just for their new account. You could check by setting a cookie when they try to sign up and correlate with subsequent attempts.
2. You might not be upfront enough about needing to verify email address. Many people assume you will just use the email for marketing and give a bogus email.