Viable on-device speech recognition is becoming reality. Apple is apparently processing some speech recognition on-device now: https://www.engadget.com/ios-15-siri-on-device-app-privacy-1...

So far there aren't any particularly good open source voice recognition models though, in large part due to a lack of training data. You can (and should!) contribute to Common Voice to help change that: https://commonvoice.mozilla.org/en