The banks don't know how much money I have, because the different banks I use don't talk to each other. The bank knows I buy things from amazon, but they don't know what I buy. The government knows what I report in taxes, but little else. Shops I purchase from know that I have paid, but they don't know my name or my bank account #.
Every party knows something about me, but nobody knows enough for me to be worried. The current system is pretty good at protecting my privacy, especially given how primitive it all is.