That was true for me in high school, but not in college.
At Caltech, it took me about an average of 3 hours to learn the material covered in an hour of lecture. A joke at the time was "one week into the semester, and I'm already 3 weeks behind!"
A friend down the hall could learn a Caltech course in a few hours by flipping through the textbook. I was nowhere near as smart as him.
I very much doubt this, and I'm curious about the truth. I suspect that if you replaced 'smart' with 'well trained' it would be true. No one is born knowing support vector machines, but if you'd previously learned perceptrons and quadratic minimization you could understand them in an afternoon. My guess is that your friend was very very precocious - and well trained.
Now I'm curious. Do you know what he ended up doing?
He's also one of the nicest and unassuming fellows I've known. He's like what people say Woz is like (I've never met Woz).
I think what happened was I "learned how to learn". Part of that indeed was getting better at picking out what to focus on. I don't believe I got any smarter.
It was still a full time job to keep up, though. 33 years after I graduated, I still have "examination dreams" where I attend a class and have no idea what the prof is talking about. Fortunately, these dreams have become less and less frequent over the decades :-) But it was worth it. I'm glad I've never been in combat, I shudder to think about the nightmares vets must suffer.