I plan to abandon nui.nvim for implementing the UI (actually, we only use nui's Split now, so it's exceptionally simple to abandon). Regarding the tiktoken_core issue, everything we did was just to make installation easier for users. However, the problem you mentioned is indeed an issue. I plan to revert to our previous approach: only providing installation documentation for tiktoken_core instead of automatically installing it for users.
As for why avante.nvim must depend on tiktoken_core, it's because I've used the powerful prompts caching feature recently introduced by the Anthropic API. This feature can greatly help users save tokens and significantly improve response speed. However, this feature requires relatively accurate token count calculations, as it only takes effect for tokens greater than 1024; otherwise, adding the caching parameter will result in an error.