Very cool! If you want to do this in a 100% on-device and private-by-design way, you can take a look at what we are building at https://snips.ai (disclaimer: I'm a co-founder)

Someone actually already did a Voice controlled drone with the platform https://github.com/trancept/snips_bebop/ :)

It runs on a Raspberry Pi 3, or any Linux, osX, iOS, Android device, and it works in English, French, German, Japanese, Spanish, Italian, and more coming soon!

We'd love to show what you are building on our blog if you want to use it :)