No one wants to ban passing them out. It's just they want to ban passing out advertising without consent. I don't think that's barring free speech in any way, is it? Also, don't littering/nuisance laws start to come into play, at some point?
What's the difference between a phone book and a similarly sized brick with "Jesus loves you -- call us to find out more" on it? Can I go around leaving bricks all over the place under the guise of free speech?