All Apple devices that support Hi Siri play a notification and display lights or a UI when it activates and starts sending data outside of the device. I don’t see what more they can do.
Of course a device that listens for an activation phrase is going to listen. And it’s never going to be 100% accurate. If you can’t accept that, just set it up so it only starts listening when you press the button.