I think it's worth mentioning some people have differing ability to memorize passively. I can thoroughly comprehend, interact with, and enjoy a book but I won't really remember details after the fact.

I can put in extra, additional effort to commit it to memory, but this is much slower, and generally removes the fun, so I've reserved this for assessed reading.

I think it is an adequate heuristic for many people that if you don't remember details about a book on your bookshelf, you haven't properly engaged with it. But I think it is not universally true.

As another data point, I also never remember the lyrics to my favourite songs unless I study them line by line.