Home Assistant has an open source smart speaker: https://www.home-assistant.io/voice-pe/

It's still in preview.