That works if meanwhile Google hasn't decided to increase the target api level requirements [1]. In that case you may not be able to just republish the app, and extensive refactoring may be necessary.
Forcing apps using old sdks out of the app store is probably the main reason they do this.
[1] https://developer.android.com/google/play/requirements/targe...