Completely get that. I always had the same issue with podcasts, which is why I built Snipd together with some friends. If you have the mp3 of the audiobook you can create your own private rss feed with services like SimpleCast and add that to our player.
For private feeds we don't yet have our AI features activated, but you can still highlight your favorite moments, add your notes and sync them to Readwise.