If you removed the empty space in our atoms, you could fit all humanity into a unit roughly the size of an m&m.
If it's an equilateral square pyramid, the volume would be about a quarter of a cubic kilometer. If we assume humans have an average mass of 65 kg and the same density as water, that gives us room for just under 4 billion humans.
Err, there are over 7B people on the planet right now.
And that's only the living people, we got lots of dead human bodies lying around, too!
The 5B mark was ~1987 according to wikipedia's population numbers in 1985 and 1990.