It's because if you only allow an app to access an approved and whitelisted API endpoint, that endpoint can trivially implement a proxy and essentially act as a gateway to the open internet.