We could just put all the books in one, big room, just aisles and aisle of shelves, everything numbered sequentially. We could even build a library as one long hallway -- one single line of books.
But we choose to divide our buildings up into rooms. We choose to create that hierarchy.
Edit: I suppose, though, that our minds evolved to operate in physical space. Go hunt some zebra, go climb a tree... things like that...