Though I don't know of any studies about algebra in particular, kids show greater neuroplasticity in general than adults. So learning anything as an adult tends to be more difficult.
But I live with an American who reasonably fluent in Norwegian after 2 years of study in their 30s. It's certainly not impossible.
I have no doubt that math, like virtually everything, is more difficult for adults to pick up from scratch.
My conclusion is that some people simply can’t learn algebra. Be it genetic, environmental, lack of intellectual fortitude, lack of “intelligence”, or any combination thereof it is simply the case that some people can’t learn algebra.
However that feels really different from saying a not insignificant portion of the population, inherently cannot.
Considering that a small percentage of careers require more than basic algebra/geometry/statistics, it's not hard to see that math has a "why should I learn this?" problem more than a "am I able to learn this?" problem.