I use a messenger installed via Flatpak. So whenever I receive a file in it I cannot save it anywhere outside my Downloads folder. So every single time I then have to open a file manager and manually move the file somewhere else. Similar for the other way around - I have to copy any file I want to send into either ~/Downloads or ~/Pictures. Not a single other path is visible or accessible, if I click on "Desktop" in the list of favorites nothing happens. This is confusing and annoying for me, this would be super confusing for a user who doesn't know and care about file permissions.
I like flatpak as it can nicely fill gaps in official repositories whitout being bound to a single distro family once again. But honestly I understand why apps would just go "allow all", because anything else drives the user insane the way it is implemented now.