I'm no historian but I don't think Newton was collaborating with any black people. Even if he was, you can't ignore the fact that the dominant academic institutions of today evolved from imperialist and white ones (e.g. the Royal Society).

That's hundreds of years of development and cultural momentum geared mostly towards people who find Latin/English natural. This massive and ancient ecosystem of information is best accessed when you can have some tea with your professor friends who only know [insert Romance language here].

There are of course other nonwhite institutions, but their development has been stunted and infiltrated by colonialist ventures.