Some do. Some mean buying something like GLD. That leaves you the risk of the exchange going out of business, though. So some prefer to actually take physical possession of the gold. That leaves you vulnerable to being robbed, though. So some prefer to have it stored in somebody else's vault. That leaves you vulnerable to the vault company going out of business, though.

There is no perfect answer. Different people worry about different secondary risks, and do different things in response.