Do you really think that's even in the realm of possibility? There is so much paranoia-driven FUD in these comments.
But let's say that yes, this absurd hypothetical is possible and happens.
Someone is capturing full audio from your device even though it allegedly only transmits when certain phrases are used (e.g., "Alexa..."). They've tunneled through your modem, router and AP and are capturing directly from your device.
That same someone is somehow able to process hours of ambient sounds, conversations and everything else to pick out someone using an inappropriate tone with their children.
They then take these recordings to a local child protection agency (e.g., CPS) and present the audio along with your information to develop an actionable case against you.
That child protection agency then decides your tone was strong enough that they need to pursue legal action.
How does this hold up in court? How does illegally-obtained audio stand as evidence? How are they able to prove it was you and not a relative or visitor?
It won't hold up; this is insane. Your point about the wrong tone being used against you is insane. I get being paranoid and not trusting these devices but get real!