But this is just an indexing issue: when you read a book, you acquire more from it than some list of facts you can intentionally recall afterward—that's just one way of indexing knowledge, and not a particularly useful one.
As an example, let's say you've read a book on compilers. Is it more useful for your brain to organize the information from the book so that when you think of certain aspects of compilers, the information is recalled; or is it better to have the information recalled when thinking of the book that it came from? Recalling the info. in connection with the book itself typically offers much less value than recalling it in connection with the book's subject matter.
You don't know about most of the knowledge you have, though it is still active in the sense that it participates in interpreting your experience and determining action. Additionally, reading (without intentional memorization) will add to this knowledge.