It seems like a pretty easy thing to do, and I don't know why its not common / not a thing on common Android phones.

Excuse me? SAF is awful and source of constant complaints from developers.

They can also request the user for a specific folder to get read/write permissions for (rather than the entire shared storage space).

Just requesting full access to the entire shared storage is a lot easier to implement, but it's also not great for privacy and security. Google is trying to crack down on these permissions in the Play Store for this reason.

Several apps have been caught extracting location history data from pictures stored in the camera directories already, so it's not a theoretical threat. Unfortunately, app developers have been reluctant to implement the (much better) specific file picker introduced in Android 4.4, but now they're being forced to if the want Google Play access in the future.

The scoped storage API has some downsides, though, such as reduced performance. Not really a problem if all you want to do is upload a picture or save a document, but when you're running executables from there you may run into a wall. This can be circumvented by copying the files to the app's private data directory, but people may want to use the selected folder to copy output back to Android, and a copy action obviously doubles the space required at least temporarily.

That being said, I believe applications shouldn't refuse to work entirely without full access to every file on the shared storage space. They can ask the user what they want to do if the storage permission is denied rather than refuse to work.

Extra unfortunate is that the new file picker Google introduced a few months ago for media selection is absolutely terrible in my opinion (no directory support, just a flat list of images), but some other phone brands offer file pickers of their own.