Until very recently (ios 13/14) apps did exactly this. There are arguments for the walled garden in terms of ensuring applications are built, signed, and distributed safely but developers (e.g. Facebook, etc) absolutely abused the system APIs to spy on users and collect data they had no right to.
Even with the newer protections, apps randomly want to access my contacts or local network or location. The average user (like my mom and dad) are most likely just gonna click through those.
I'm not arguing for ios/android here but we should be factual. I will say I like the walled garden for other reasons, such as a consistent payment gateway, etc.