Similar to the Monty Hall problem [1], the mistake you're making is calculating the opportunity cost at the time of sale, rather than within the context of perspective. Opportunity cost is dynamic, not static.
If I approached you in 2012 with two envelopes, one with $20 worth of US bank notes, one with $20 worth of Bitcoin, you'd most likely take the cash. If I approach you again in 2021 with the same envelopes (unopened) from 2012, only a fool would take the cash.
Speculative assets would not exist if your theory held water.