Fermi's paradox says nothing about how likely aliens are to visit earth. It notes that we can't see any sign of them when we look into the sky.

Perhaps the humans don't care about the ants, but the ants damn well can see signs of human activity.