With gold I trust the laws of physics and reality (not a human based system), with BTC I trust the consensus of the network participants and that the rules everyone follows will result in trustworthy outcomes, with a bank I trust that bank.
The big threat to BTC trust therefore IMO is if the network participants as a whole or enough of them aren't trustworthy. Marginal cost of mining a block favors cheap electricity countries (i.e. hydro China power) in the long term as the difficulty of the network changes to price out other competitors as they scale up. In that instance do you trust the person with the cheapest electricity to verify your transactions? Given my understanding of crypto that's probably the biggest threat to any long term investment in BTC. For PoS currencies I'm guessing your trusting the stakers by amount put down (i.e the rich people) to verify your transactions.
Of course, the counter-counter-point to that is that the Bitcoin network also has "intrinsic value", and it's that it enables trustless/permissionless transactions to people that might be engaging in illegal commerce or are trying to escape the currency of a regime with poor monetary policy (Venezuela, Bolivia, Nigeria, Zimbabwe, etc).