For us nerds in the Portland/PNW area, the Rice Museum out in Hillsboro—despite a name suggesting it has an exhaustive display of rice varieties—has a terrific collection of large and unique mineral specimens.
https://www.ricenorthwestmuseum.org
Mines Museum of Earth Science (Golden CO) and The Harvard Museum of Natural History (Mineralogy room, Cambridge MA)