Jasper is another cool project that has a bit better of a foundation. You can make little Python modules for pretty much anything.
https://github.com/cantino/huginn
or Node-RED for JavaScript (this one is backed by IBM and has a neat drag and drop UI)
I think once the voice processing is done, the hard part is getting it to actually do something useful.