Agree - I don't know why this isn't the default, or why "all photos" needs to exist in the first place. The only valid use case I can think of is for selecting an image from a gallery in-app.
Other examples where all photos are required, if maybe not valid: third party automatic photo backup, showing a random photo on a widget, prompting a user to post a photo of a location to Google Maps automatically…
It should be possible to allow access to all files, or allow picking a single file. The weird intermediary where I need to first select that I don’t want to allow access to all files, then select the files I do want (the single file I’m posting/sending), then select that file again from the app’s custom picker from amongst the files I’ve previously shared with that app, is absolutely awful. Steve is rolling in his grave.