That's the point. Americans don't usually call Spaniards and Portuguese "Hispanic" any more then they call a Blond haired, blue eyed person born in America, but who's parents were born in South Africa "African American", which is to say rarely and usually in a comedic or anecdotal manner.

When he's referring to "Hispanics and African Americans" he is trying to refer to specific "racial" (insomuch as the concept exists) groups.