Rate limit each unique source IP address would suffice?
Also, couldn't you generate a random six digit number and ask a user to write it as a workaround for using Google? Or use a self-hosted service?
The first version that I made ~6 months ago had basically zero protection against spammers/trolls/etc, and it's been a non-stop arms race since then. I could rant for long hours about the numerous types of attacks and what I've had to do to prevent them.
I like the random code idea. But you'd have to ask more than once, otherwise spammers could go through the code once and then turn on their script. So it again becomes a tradeoff of how much to worsen the experience for normal users