Nigerians have lots of APOE4 (a gene that leads to much higher rates of alzheimers), and really low cholesterol. And they also have lower rates of AD.
I think base on current evidence your family might just have great cholesterol genes and bad AD genes.
Btw how old was your father, aunt and grandmother when they got AD?