For TUIs, Rust/Go vs Typescript doesn't really makes a huge performance difference and you lose the 50x bigger community advantage of Typescript.
TUI renderer is the one using the memory heavily so your terminal takes the heavy lifing. If you're managing the buffers and out-of-screen context good enough, Typescript can be pretty efficient.
It’s not about the terminal at all which as you noted accounts for minimal isage. It’s all the internal chat and history and everything else the agent tracks - all of which are smallish (and largish) strings allocated on the heap.
I don’t have the same issues with rust based tuis.