You're only going to be able to do wake-words and command recognition locally on a microcontroller. The maximum external SPI RAM on an ESP32 is 4MB.

https://github.com/espressif/esp-skainet

https://github.com/tomzbj/dhrystone_score