The difference is securing bitcoins in high-risk environments: live automated servers. But that only goes for a small minority of bitcoins as only about 0.5% of bitcoins are traded on a daily basis, the rest can be in what is called cold-storage which is way, way easier to secure than a bunch of physical money, gold, diamonds, whatever. The 0.5% is a bit like the security of wallets in people's pockets: not very good and not comparable to bank-grade security of those same people's savings in a bank. Or comparable to lone ATMs which can be raided.
Beyond that, bitcoins exist on an open ledger. While it may be hard to trace that now, we're seeing blockchain analysis tools develop all the time. Who knows what they might uncover later? I certainly wouldn't be excited as a criminal to leave a permanently recorded trace of every single transaction ever since the moment of the theft, with the prospect of decades of cheap supercomputers doing analysis of this open source data set which might reveal yourself. Bitcoin is great for pseudo-anonymity when buying por n online, but it probably won't stand the test of time in protecting you from every criminal investigation, a bit like a randomly generated pseudonymous email address gives a large extent of privacy, but not if you're a criminal.
1) Don't let anyone find the private key
2) Don't let yourself lose the private key
The more of 2 you do, the harder 1 gets and the more of 1 you do, the harder 2 gets. If you have only a single copy on paper, it's hard for anyone to find, but also easily damaged or lost. If you make a backup on a computer, a virus might find it, making the paper copy worthless.
Not only that, but from the moment you generate the key, you have to be completely confident about the security of the computer you're using. People recommend a clean install of OS from the original CD and never connecting to the internet. That's not impossible but not easy, furthermore you have to carefully destroy all record of it left on that computer. So unlike gold, not only do you have to keep it secure in the future, but you have to have made sure that it was always secure throughout the past. You can't transfer ownership to anyone else without them going through the same rigmarole too, unlike gold which you can physically give to someone and they can be confident it won't evaporate in front of their eyes one day.
I agree with you about blockchain analysis. Who knows what new techniques might be able to dig up. They might not have much certainty but perhaps enough to put suspicion on someone.