And for others it goes quite quickly! There is no doubt that programming can be helped with a natural aptitude and that it won't make sense to everyone no matter how smart they are, but that is true of many fields.
I'd also argue it's often a teaching problem. I'd highly recommend this essay that goes into the flaws in particular with introductory CS education:
https://felleisen.org/matthias/Thoughts/Developing_Developer...