Keeping them in lots of separate places just magnifies the probability of the public not being able to view them.
That's not often the case. These museums were built to house these collections, and often include rooms full of drawers, compacting shelving etc with the collected objects. This incudes the basement, but not only the basement.
For example, the Natural History Museum in London has "the majority" of their 80 million specimens stored in the museum in London.
You may be able to go on a behind-the-scenes tour of your local museum.