IMHO this is justified since people often overlook how critical an attack vector this would be.
Imagine if a state-sponsored "hobbyist" posted a pet project like the OP to Reddit and started harvesting keys/password reset capabilities for huge amounts of users.
Hell, people flocked to use unroll.me and then it turned out they were (predictably) scraping your entire inbox and selling the data to advertisers.
I don't believe that worrying somebody's skepticism might hurt feelings is a legitimate reason to downvote (remember, downvotes aren't about whether you agree with somebody; it's about whether they add constructively to discussion).
Aside: Great work OP. Seems like an incredibly hard sector to make traction with though, so best of luck.