Very interesting Ben! Did you have a look at https://wit.ai ? It's an API that encapsulates both speech rec and semantic interpretation.