If the original claim was merely that a certain percentage of the population are not going to learn algebra - it would be uncontroversial and i would agree with it. After all, we all know that some people fail classes.
If the claim was merely that math comes easier to some than others - again it would be an uncontroversial claim and i would agree with it.
However, my reading of the original comment was that a good portion of the population literally are physically incapable under any circumstance, including the silly hypothetical circumstances i laid out, to learn algebra. I think that is false. It sounds like you do too. So i guess we're in agreement.