It sounds like you already know the answer, considering you said you'd ignore anyone who says you should give up. :)

Of course you should forge ahead with it. Your best bet is to probably to go with your gut. Look for a programmer to partner with and in the meantime, maybe grab a few books on programming so you could help (even if its with menial stuff).

Good luck!