https://openaudible.org could be considered somewhat Calibre-like. If it's more about self-hosting, media servers like https://jellyfin.org may be more interesting (although I wouldn't call those similar to Calibre).
ffmpeg -activation_bytes XXXX -i audiobook.aax audiobook.mp3
It also supports decrypting but you have to pay for that.