You weren't dumb, you were ignorant and ignorance can be overcome and more likely so if you are high IQ. In asking, "why can't you do x?" what they were really saying was, "I know this child has the capacity to understand this material but I don't have the power to help them do that." It wasn't about you. It was about them.
Children don't understand that. Children internalize everything. They blame themselves and you blamed yourself.