Identity of the author matters because a writer's work is informed by their experience in the world. As a white male, I will view & experience the world differently from Jane Jacobs. Even in a book about facts, a person's relation to those facts reflects their lived experience.

This is especially important in fiction books where you are immersed in the experience of main characters. If the authors identity mirrors yours, then the experience of the main character will not build my understanding of how other people live. At this point, you are reading only for entertainment (which is fine, but you must acknowledge and accept that).

Books, whether fiction or nonfiction, are stories. They present information in a narrative. That narrative will be determined by who writes it and how they've lived their life.