I've been verbally assaulted (proper screaming-levels of verbal attacks) for being a pedophile because I'm at a park playing with my own daughter. I've had groups of mums surround me while they call the police, and force me to wait for them to arrive. I've very regularly had people pull their kids away from my kids and not let them play together, while all the other kids with mums there are allowed to play amongst each other. I've been told I'm "assaulting" my wife by "forcing" her to work and miss out on the all-important motherhood experience. Doesn't matter that mum works in ICU/surgical saving lives, and absolutely loves what she does (and works 3 days a week, so gets plenty of mothering time). I've been followed for blocks and blocks by people videoing me on their phone saying they're reporting me for kidnapping. I've been told to leave the park more times than I could ever count, because "it's a place for mums and their kids". I've been rejected from library story time and family centres because I'm not the mum. I've had the police turn up to my door because neighbours saw me tending to a crying kid in the yard (fallen off the bike) and the police were told I had kidnapped them and was assaulting them... I could go on for many more paragraphs.
In all these situations I calmly explain that I'm the father, and just try move away from the people or leave the park to save my kids having to experience this stuff.
People are fucking crazy.
Edit: To clarify, I'm in Australia. I have also run into various other stay-at-home dads who have similar stories. It does seem to be more in the inner city suburbs, as we move a lot and I haven't had it nearly as much in more rural areas (though it does still happen).