I'm not disagreeing with you, but the comparison is breaking down here. Reading a book is the specific intended purpose. It's why the book exists, not just fair use. If you couldn't remember what was going on in the book as you read it, it would be worthless.