With exceptional interest and support, certainly anyone can absorb all these concepts.
Of course if we take the case of someone about to die in 24h max due to brain cancer, then sure we just don't have the knowledge and resources to successfully make someone acquire that kind of knowledge.
But in the general case, people not learning advanced math notions is everywhere in the intersection between individual having no interest and society not pushing them in that direction through resource allocations.
Also since it's about Hardy we can not withdraw the case of Ramanujan. Yes, there are people whose brain is wired in very uncommon way that push them toward exploring uncharted territories where few have interest and even less have the ability to go through and survive. That said, once the path is paved and everything is in place to accommodate the accessibility of the place, there is no longer the same level of struggle to be expected.
More often, we lake the great teaching resources, rather than the sufficiently apt learners.
It's arguable that no such thing as talent exists. That when we say "talent" we are really using a short-hand for a kind of knowledge and experience. I suspect that Hardy would strongly reject such a claim, based on reading his Apology.