This stems from the concern (valid, in my opinion) of preserving the collection. If you let just anyone enter the physical library to access it, your risk would be higher that someone would accidentally damage or take some of the collection.

For what it's worth, my understanding is that the Library of Congress has similar restrictions for accessing at least some of their archives.