I would think that this is a bug in the ringer and not an overall limitation of the Android permission system.
No user would point to Google and say »VLC doesn't pause when the phone rings, fix the ringer, please.«. Rather they point to VLC and say »VLC doesn't pause when the phone rings, fix VLC please.«
Split your APK based on SDK version. Have different manifests with different permissions & code for the respective versions. The new Gradle toolchain makes all of that easy to do, but it is tough to maintain lots of version specific code long term.