As far as my understanding goes, space is very unevenly distributed, with lots of debris and asteroids in dense (well, humpf) parts of space, but the vast majority of it is just... empty space with so little matter in between that it is basically negligible. To mine an asteroid likely means going to where it's more probable to find a suitable one, and also more risky. Asteroids around us are few (IIUC).