The crux of Bitcoins is that its value is derived from people's belief in its value. Lot's of assets and to an extent fiat moneys work this way. The "magic" of bitcoin et.al. is that they are finite, have clear chain of custody, and can be traded peer to peer. Those features can and do inspire confidence, but at the end of the day these coins have limited real world utility outside of money transfers and holding wealth. Those uses are directly tied to the faith of the market for those coins.
If someone asked me if they should buy and hold these coins, I would say no. However, speculation is somewhat in the eye of the beholder and turns on the risk of losing the investment. It's nearly impossible to predict future changes in asset prices, especially when an asset has been steadily increasing in value for years, but has a limited history.
TL;DR you're misusing the phrase "Ponzi scheme" which means something else.