Study of sapiosexuality suggests some really are attracted to intelligence
psypost.org
psypost.org
Didn't look at the details, though.
People look at attraction in too much of a binary way. I would think it's most likely that humans look for both physical and intellectual prowess in their mates. There might at times be a bit of a trade-off between the two, but likely we're trying to optimize for getting as best we can in both categories.
My first marriage was much the same way. I didn't articulate it at the time (there were other problems), but he lost my attention... not because he couldn't keep up, but because I got tired of never being intellectually challenged. (He was and is a nice guy, just not the right one for me. I should have realized this when I discovered he owned one [count it one] book to be read for pleasure.)