Nice, Kokoro really is the jump that made long-form listenable without the legacy-TTS fatigue. I went down almost the same path: hated the e-reader TTS, found the newer models, and ended up building a hosted audiobook service (tomevox) on a similar GPU pipeline.
Two things that ate most of the time once the TTS itself was good enough, in case they save you some. First, pronunciation of names and invented words: a per-book lexicon helped more than swapping models. Second, chapter-boundary handling for clean M4B chapter markers. Raw generation ended up being maybe 30% of the work; manuscript prep and a human QA pass before delivery were the rest. How are you handling pronunciation overrides?