Yes, they most definitely should. A video that specifically targets babies/children should not have ads that are 18+.
Where else in the world would you view media for children and be shown ads for drug dealing dramas?
But to be honest, maybe this is an opportunity to create some kind of maturity level standard, where content shouldn't be above a set threshold, regardless of source. On YouTube For Kids, it would be lower than regular YouTube, which would be lower than RedTube. Advertising would be compelled to build in functionality that checks a page's threshold and serves ads that match.
I'm not saying I agree with the idea, just putting it out there.
That's what every advertising platform does...
It doesn't matter that some adults would watch also. You would never in a million years see an ad for The Wire while watching Sesame Street or Mr. Rogers, etc. Pretty obvious why not...
The same standard should be applied to "new" media.
We simultaneously what internet companies to do exactly the right thing but don't want to allow them to collect the data needed to do that right thing. It is an ironic behavior I notice in the tech community.