We don't have to automatically trust them, and challenging is important. I basically agree with everything else you said, but I also don't think it's that big a deal outside of edge cases.
My main issue with this story, and stories like it, is that parent's aren't arguing "we need the library to stock more books which present my beliefs in a favorable light".
They're generally arguing "I don't want books available to any child that present anything I don't personally agree with."
So, yes, absolutely, we should be cautious about institutional ideological blinders.
But I'm under no obligation to take serious anyone whose response to those blinders is to limit access instead of expand access.