I wonder if what makes humans "intelligent" is exactly this. Plus we have a 70-90 years of life. Which is long enough to learn and do stuff, compare that to 10-15 years for your average crow.
We start bombarding babies with information since a very early age. Parents, are incentivized to teach their babies how to talk, walk, read, write, do stuff, eat, sing, etc... The amount of cognitive exercise is insane. And a human will only be able to "say" something useful until he is 7-8 year old. He's kinda of a "retard" before that. That's 7-8 years of training just to get started.
Then humans get bombarded with education: Math, Physics, Language, Writing, Sports, etc... And they are a strict about going to school and performing well. That would take another 12-15 years of your life to, hopefully, learn something useful to society. Add to that 3-4 years of learning in the job, and a human is only able to bring food to the table after 25-26 years of learning and training.
That's a hell lot of time. No other animals in the wild are given this chance. Let alone their environments and their physical capacities are taken into consideration. We judge animal intelligence by comparing it to our self-architectured modern environment.
tl;dr: Humans might not be smart after all. It might be that we have been lucky that our ancestors have started the ball rolling and we have had enough time during our lifetime to make up for the initial investment of learning.