Bitcoin does not offer any form of privacy as a direct result of being decentralized. In many ways bitcoins are a huge step back from a privacy perspective simply because private people can follow all transactions.
Your bank offers the illusion of privacy. Sure, a random person can't just walk up and see your balance. But a teller can pull up your account and look around. Sysadmins can certainly see your money pop up in server logs. A particularly motivated individual might even be able to pull data from disk images or memory if they were so inclined, so how safe is that information really?
I'd argue that knowing everything is public is better. It motivates people to come up with solutions like Zerocoin and CoinJoin.
As for decentralization vs centralization. If banks asked me to sign up with my public key and didn't ask for my name or address or identification number, then sure. But the fact is that they do. The fact is that they are infiltrated by the NSA, et al. A direct result of decentralization is the absolute need for privacy, otherwise the system breaks down.
Zerocoin will completely fix this.
IP address is one of many ways that particular bitcoin addresses can be linked to real identities. The rest is graph analysis.
I buy a bitcoin, I send it to BTC-e.com and sell it for $800. Now I have an $800 USD balance on my account. I wait for 4 days then buy Litecoin and send it to my offline wallet.
Now, how would you track that? Note in my HN profile I have a BTC & LTC address that I've done exactly that with more than once. I would love for someone to show me the blockchain transactions that prove that's what happened. Alos note, this is pretty much how the SheepMarketPlace thieves got away. Some guy tracing the bitcoins until it was revealed that he was just following a BTC-e.com's internal operations address. Also note that you don't need to give BTC-e.com anymore more info than a valid email-address. Just get your coins from some other place like mining or whatever, send them to the deposit-address given to you on BTC-e.com.
For people not familar with bitcoin, consider this real-world example. I steal a $100 bill from your wallet but you have a paper-thin tracking device inside it. So you watch it physically move around in the world. Now, if I go to an Airport and exchange the $100 for 89 EURO... you've lost track of the original thief. The thief still is holding the funds he gained from the theft and you're just following a $100 around in the Airport exchange system that may end up anywhere. You know where the $100 bill is but you'll never find the thief. Mix this with stuff like localbitcoins.com(equivalent of just finding a random person on the street that will exchange currency) and you'd have a pretty tough time finding out exactly how the original person spent the funds.
I guess nothing is completely out of the reach of authorities if they really wanna get you, but it's a bit more difficult with crypto-currency than just looking at your bank account history. Now, if every unregulated exchange turns into a bureaucracy-controlled entity then I guess they're no longer much better than a bank.