I had the most success using Dragonfly with the Kaldi engine, which handles the heavy lifting of speech recognition and provides a nice API on top of xdotool for desktop interaction. You can also check out Talon, although I think its Linux support is still somewhat experimental.