They should feel silly. It's not the market value of the spent Bitcoin ($20), but the opportunity cost (~$34k).
If I approached you in 2012 with two envelopes, one with $20 worth of US bank notes, one with $20 worth of Bitcoin, you'd most likely take the cash. If I approach you again in 2021 with the same envelopes (unopened) from 2012, only a fool would take the cash.
Speculative assets would not exist if your theory held water.
If Monty Hall asked you if you wanted to change doors before opening one the problem would not be much of a problem.
If you buy anything at $X with intention of "hodling" it to increase it's value, unless you have a plan (i.e. I will sell at $Y) then you are just holding on to nothing. You will either sell too early or too late (now talking about the absolute maximum gains) and then you will always "feel silly".
Unless OP actually put in, say $300k (a price of a small house) and that was stolen he didn't lose "a house worth of money" since he had no exit plan. I don't play crypto, but I could still say that I lost $600 when the market crashed couple weeks back conveniently forgetting that my initial investment was only $10 back in the day. I could even be more dramatic and say that I lost half of my bitcoins just in the crash and not mention that I only had $1.2k.
I don't think that's holding on to nothing just because there's no exit plan in place.
If I invest 30k in index funds that are managed by my bank, with the expectation that the value will rise and thus beat inflation, i wouldn't say that I'm holding on to nothing.
Similarly, if the value would rise to 35k over time, but something (like an economical crash, or any reason) would have me withdraw my money before that, I think it would be fair to say that my actions made me miss out on the total value of 35k.
I don't see how the case of Bitcoin is that much different - most people won't have sold at its highest point, but the average value has indeed increased bunches compared to when it was not popular.
I don't think you always need an exit strategy for every investment you make - wanting to diversify and perhaps beat the inflation seem like valid goals, regardless of whether you're putting money in index funds, are buying BTC, or are burying pots of gold in your back yard.
Volatility and risks are probably another story, though.
I.e. you would have to not only account for losing $34k but also lose nearer a few hundred thousand because you could have sold at each peak and bought at each trough.