People without LinkedIn accounts? Are you quite certain? I worked at LinkedIn and know exactly how People You May Know used to work (I've looked at the code). Unless they changed something in the last couple years (and given their dev cycle, I doubt it), every single person who it recommends has signed up to LinkedIn. Of course, I heard people make these claims while I worked there too.

Also, it really isn't that hard to guess how it works (hint: are you sure you need every single person to give address books access to build a graph?).