The magic is still there! In this case, the model for OpenAI's Whisper, which is arguably doing the bulk of the work here, is Open Source (under the MIT licence), and freely available for download at https://github.com/openai/whisper. You can run it wherever you want, though something with a GPU will let you do 5x realtime (or better!) transcription.
Someone is out there building and maintaining the APIs
One hopes. If they decide to stop providing/maintaining the API, you're out of luck.
Just using APIs also misses out on the learning, community, and collaboration of open source software.
agree with parent re: stitching together -- but also, someone's gotta program the API!