I agree. Letterboxd makes this easy to accomplish with movies, and there's enough people covering videogames out there that you can usually find someone to dial-in on your taste (or find a Steam curator that lists a lot of stuff you love). TV is a bit harder, but you may be able to find a critic with a consistent voice that matches your own through Rotten Tomatoes (not the aggregate score alone) or maybe a podcast about TV where you learn their preference dial vs. your own.

But books? The level of investment to make these determinations is larger, and there's less people doing it. Or maybe I just haven't found the answer yet.

I also don't think it helps that people either like a book or immediately say "bad writing" or "too YA", which then make others that disagree look like simpletons just for disagreeing. Creates a hard place for dialogue.