People need to let their kids out more, but common, a 3 year old at an intersection? That's not what this thread is about. Even in your story you were trying to supervise them and weren't far behind.
Supervision is not an all-or-nothing thing. When someone calls the cops on your 10 year-old climbing a tree in your front yard or the neighborhood woods (let alone in your fenced-in back yard), that is ridiculous. But, there is a problem when you equivocate a toddler on a public road and a grade-schooler playing outside. One is able to stumble into problems and solve them while building confidence--the other just learned it's name and how to put on their shoes and pedal the bike.
Like, all 3-year-olds are different, perhaps his was closer to 4. Perhaps they were developmentally way ahead of their peers. Perhaps this was the most rural quiet and safe road ever. But without knowing that extra stuff, I think it is reasonable to voice your concern after finding a 3 year old on a road without real supervision.