The hard part is disambiguating customers (which is why loyalty/discount cards are so great for them), but it is really horrifying the amount of credit card history, demographic data, grocery store purchases, prescription information (seriously) etc etc that companies can collect, cross reference, and sell while staying within the law.
http://www.google.com/wallet/privacy.html
I quote: "Transaction information - When you use Google Wallet to conduct a transaction, information regarding the transaction may be stored in the Google Wallet Application. We may collect information about these transactions from the Google Wallet Application. We also collect transaction data from your use of the Google Wallet Service. For example, if you use the Google Wallet Application to make a purchase at a merchant or download a merchant coupon, we may obtain information regarding that transaction from the Wallet Application, from the merchant and/or a partner, as applicable. The information may include the date and time of the purchase, the store location, the amount of the purchase, and the offer associated with the transaction. "
Here's a thought experiment: my Android phone has a camera in it. Do you also believe that Google is collecting every picture I take without my permission?
They could have made some arbitrary statement like "by using the camera app you give us access to all of the pictures you ever take, so that we can back them up for you."
On the other hand, if you don't feel comfortable with google being the clearinghouse for the information...super? Use something else (and spur competition!). The OP's statement seems as monumental as which brand of credit card company he uses (and somewhat willfully blind in light of the information they can and do collect...much more than appears on our credit card statements).