Off the top of my head, isnt that just the -x flag?
exec $HOME/d/youtube-dl/youtube-dl -t -x --audio-format=mp3 "$@"
(Edit: I'm sure this script does a lot more, but making ytdl download only the audio is fairly simple. Note the above command converts the file to mp3 which some people may not want to do because it's lossy.)$ yt-audio --your-arg1 --your-arg2
This will sync playlist/audios configured against these arguments.
OTOH, if you specify a format directly without -x, it may not extract the audio alone. For example if you wanted M4A, you'd specify -x on top of specifying the format explicitly.
edit: as krick points out though, it doesn’t seem videos generally have an mp3 format, at least on YouTube.