I tried a few of the open source available assistant-like systems on my RaspberryPi and found that none of them is even half as good as Google's or Apple's, which is expected.
After thinking about it though, I found that I don't need the voice recognition at all really. What I really wanted is a device that can help me do a few things well, mainly for my case, listen to the radio, announce calendar events, and train time (Stockholm in my case), so I just built that into a raspberry pi with a tiny screen and a few buttons. This little device is more than enough for my case.