I've recently read and greatly enjoyed a historical fiction novel called "Augustus" written by John Edward Williams and published in 1972. On the surface level, it's about the events of the life of Augustus Caesar (better known in the book as "Octavian")—but on a deeper level, it's about the rarity of longtime friends in life, and dealing with aging and one's mortality. I put the novel off for a year because I thought I had to read a non-fiction historical account of Augustus's life first, as I thought I couldn't appreciate the novel without doing so, due to the unfamiliar character names and events. But one day, I just decided to try it out—and I found myself naturally remembering the character names and events without special care in reading the novel.
Similar experiences have been reported by people engaging with various forms of media. I've seen readers take copious notes on the novel "Infinite Jest," which has a reputation for being a difficult read, only to burn out. In contrast, readers who have finished the novel said that they didn't need to take notes, and that the story began to make sense simply by reading more.
I've also seen a similar pattern from subjects as academic mathematics, where some learners spend too much time on textbook explanations instead of working on the textbook problems, to subjects as relaxed as computer role-playing games, in which some players end up dropping these games due to a perceived need to take notes to understand the story, before they can get immersed in the game's world.
I think a lot more understanding and enjoyment of various subjects can be attained by being comfortable with confusion for a while. While note-taking has its place in understanding a subject, I've personally found that immersion is the most important factor for understanding.