You can get pretty far with the ESP32 Box though the built-in speaker is a bit tinny.

https://www.adafruit.com/product/5835

I've got it working with local inference though some people use it with OpenAI.