I ended up using it for more general purpose things because being able to have a hands-free phone call with an AI turned out to be pretty useful.
It's offline now, but here's the code with all the stack and deployment info: https://github.com/kevingduck/ChatGPT-phone/
Edit: forgot to mention this was all running off a $35 raspberry pi.