A lot of them have the majority of their collections offsite in warehouses. The "museum" is a sort of front office meant for visitors, but visitors only ever see a tiny fraction of the collection.

"A museum" is already a lot more than visitors can see. As stated they generally only show a tiny fraction of their collections and besides that many museums employ dozens of full time researchers who use the collections to do analyses and publish papers. It is as prestigious to be faculty at the Field Museum in Chicago, the American Museum of Natural History in NYC, or the Smithsonian in DC as it is at a major university. But without visitors it is hard to bring in money either directly or indirectly.