I don't know what kind of answer you expect ... of course if you're "in the zone" you won't be paying attention to an audio book. The higher concern for me isn't that you'll miss important parts of the book, but rather that you'll make errors while working or be less productive.

Try listening to it when you're driving, working out, eating, etc.