I think this is an under-rated benefit of reading. Would you feel like you failed if your mind wandered during a guided meditation? Can reading serve a similar function?
This is one reason why I gave up trying to significantly increase my reading speed. Blazing through a nonfiction book at 700 WPM might make you feel like you've learned a lot, but you probably haven't really digested anything. Do you really want to uncritically ingest a nonfiction book? I've listened to enough episodes of If Books Could Kill (amazing podcast, btw) to no longer have any desire to quickly "absorb knowledge" from a book, with how likely it is that that "knowledge" is flawed in myriad ways. I now think it's much better to go slowly, ponder what you read, take time to let your mind wander (it's connecting the rest of your life to what you've just read), and be critical and questioning of everything you read. If a book isn't worth taking this time on, it's not worth reading.
And for novels or reading for fun, why would you want to rush through it? It's for pleasure: go at whatever speed gives you the most pleasure. It's not a contest.
You're not alone there.
From personal experience, I find it much easier to get absorbed reading books like I used to when I'm on a detox from the tyranny of the screen. It doesn't take long at all; a day or two and I start to feel some recovery.
Taking even that much time away feels increasingly difficult... But like going to the gym, you rarely regret it afterwards.
I was amazed.
The trouble turned out to be it simply got harder and harder for me to read, and so I enjoyed it less, so read less. Getting progressive lenses fixed that.
Earlier this year I went to the opticians (it'd been a few years) and lo and behold I had a prescription for the first time. A very mild one, and I suspect it's the astigmatism that affects reading more, since it makes smaller fonts sort of 'fuzzy' around the edges, even though everything seems perfectly fine at that distance (or any other) without the direct comparison.