Moon+ Reader has offline playback although the voices aren't as good, but on the bright side, if someone makes a local AI text-to-speech engine, then that can plug into the app and it'll work fully offline.
I found a single file didn't make sense as you lose your place in it easily.
Instead splitting each chapter into a file seemed to do well.
But, you really do want a listening app. Otherwise it get harder to share and listen to on your phone.
So far I created a listening app as a PWA for the time being.