I would be much more concerned about some well intentioned rando calling the cops on you because there's a child unattended.
In my neighborhood the street is literally full of neighborhood kids almost every day, ranging from 2-3 years old to teens, out riding bikes and playing games. There's a park a few blocks away that my kids (3yo-10yo) walk or bike to unsupervised.
Obviously as a parent you still worry - for me I worry more about someone getting hit by a car than being abducted - but it's nice to know at least that child services isn't going to show up just because the kids are out playing.