Probably because if you get API access to someones email account it is game over. And people are stupid so some of them are going to click yes to some scammy app. And then they will blame Google for not protecting them.

Because otherwise tons of people anonymously create api keys with extremely wide scopes for small / low quality apps.

When those inevitably get used for nefarious purposes; Google image suffers as a result.