The OS also doesn’t know when you uninstall an app whether the app is really gone. You could have copied it to a network drive first, or maybe you have a different version of the app in a folder somewhere. Who knows?
Apps that install persistent services typically check if the .app bundle exists in applications so in most cases just trashing the .app is enough to stop any errant left overs from executing.
What I mean is that you may have moved the app to a USB drive or something, and so the stuff in ~/Library should remain.