The nature of the Bitcoin's SHA-256 POW means that individuals can no longer meaningfully contribute to the integrity of the chain, and this responsibility is increasingly held by fewer individuals with access to (relatively) expensive specialized ASIC machines.
The Bitcoin system has (relatively) few legit trade opportunities yet, but is well used by scammers and black marketeers.
The median person has little use for Bitcoin as it is now.
MtGOX showed that exchanges are a point of vulnerability.
Silk Road seizure showed that Bitcoin is vulnerable to rubber-hose cryptanalysis and reminded everyone it is only pseudoanonymous.
Governments have already moved to attack Bitcoin or users of Bitcoin. Mostly these are proactive attempts to defend domestic currency controls, or concerns over money laundering.
The "dust DoS" attacks have called into question whether Bitcoin can scale much larger than its current extent.
You can't use Bitcoin with Apple Pay or Google Wallet. You actually can't get an app approved for Apple's App Store if it includes the ability to send Bitcoin. Google play doesn't allow in-app purchases in Bitcoin.
There's no reason to ditch it if you have already adopted it, but there's also no particular reason to adopt it in the first place, if something better might come along within a year or two. The banks aren't buying in. I haven't either.