Thanks for the grace there, and that's a problem we are solving. You might have X contacts on Circles but you're only going to see from those you have placed in a circle. So only content from people you actually want to see from.
In terms of syncing contacts, we don't store any contact book information on our servers. We pass through the contact book to see if a user exists there, we return them back as a connection. If not we return them as someone you can invite.