I like your box analogy, but I don't think it goes far enough - with C++ pointers it is not always ok to put something in the box. If the pointer is out of date or invalid there is a chance that writing to it will make the OS kill the entire process.
Languages with garbage collection don't tend to have that kind of problem, since whatever is at the memory location will stay at that location until everyone surrenders their pointers.