If you're iterating over all your points calculating distances, you are going about this the wrong way.
(edit) The author could use my handy python quad tree if he so wishes -- https://github.com/Dav3xor/pyquadtree, and if he asks nicely, I could even add support for simple approximation of spherical coordinates.