Why is it on Google to stop this and not the banks?
Yes, banks should (and sometimes do) double- and triple-check with you before allowing large transfers/withdrawals, but scammers know how to coach their victims past this. Speaking from experience.
(I also don't fully agree this is Google's responsibility, and I am not happy about this development. But there are legitimate points in favor of outsourcing the question of "will this software do nefarious things" to some kind of trusted signing authority.)
how would the clueless victim check anyway?
There are more grandmas who just want their banking secure than there are FOSS advocates wanting full system access.