As I understand it, Apple devices don't start sending audio data to Apple until you say "Hey Siri" (which is interpreted on the device). It may not be as explicit as pushing a button, but it seems close.
And most important, you can disable "Hey Siri" trigger on your iPhone/Mac or Apple Watch. If you disabled "Hey Siri" trigger (and left "Siri" on) then you have to long press "Home" button (or button on the right, if you are on iPhone X and later) to start "Siri"
I know, but that prevents CarPlay from working, which I really like...