In 1980, the US was about 80% white demographically. Today it's about 63% white and falling rapidly. The difference between those two, is tens of millions of immigrants from Latin America and Asia. The US has absorbed 10 to 12 million immigrants from just Mexico since 1980.
Remove the immigration from Latin America + Asia and US population growth over the last 30 years in particular falls to barely a trickle; it'd likely be flat at this point.