Why aren't you using the compatibility libraries?
The whole approach of backwards compatibility has not worked out well. Google's alternative approach for the last two years have been the Play Services, which are now a behemoth requiring multidex in your app, when the majority just use the API client libraries from them.
[1] https://developers.google.com/android/guides/setup#split
This puts future development on two, or, for very mature products, on three tracks where you have a code-base for up-to-date devices, and another for somewhat training edge devices, and maybe a third for old devices. Very often new phones have an outsize value to app developers because that's where the new phone customers, who buy apps at a higher rate than people with two-year-old phones.
YMMV. If you don't have tens of thousands of users, you aren't going to do it that way. Or, if you app has no need for the latest features, you won't do it that way.