>virtually no one disagrees with "the ability to learn grammar is hard-wired into the brain".

No, this is exactly where the field is split. Chomsky says that if a child socializes with people speaking a language with each other, the child will learn that language without teaching, punishment or reward. I.e. the human mind is not a blank slate buy has innate capabilities. Ideologically, this is an individualist take on linguistics.

The other side says that children have to be taught to speak a language properly and that language is difficult. Ideologically, this is the collectivist take on linguistics - children are blank slates to be filled in by society.

The people that have a problem with Chomsky's ideas are authoritarian collectivists. It's not about science.