There is no reason why Evince/Okular or mpv (to name a few apps which handle files with complex formats from untrusted sources) should have the right to access anything beside their ~/.config/<application_name> and the file they are currently viewing/playing, or maybe read-only ~/Music. If you want to do a "Save as", you will do this through a OS-controlled file dialog, or save it to /tmp and copy it.
This can be achieved with AppArmor, with a caveat that in X, applications can steal each other's windows, but unfortunately this is not the default and easy configuration on most distros.
- autorun and keyboard shortcuts of your window manager -- one can hook an evil command to Ctrl+C
- ~/.mozilla -- you can add arbitrary javascript to your profile or extensions
- any application which does not expect to have its config externally tampered with and this may result in various errors including RCE
- ~/work/FooProject/Makefile, configuration of your IDE (which contains list of commands that shall be executed to compile)
etc.
An explicit allowlist would be a better option, IMHO perfectly manageable - with a popup window "the app wants to access <file>, allow once | allow permanently | deny".