I’ve been tackling that recently.

In my case my nuc and my “terminal” are connected via 1Gbps Ethernet. Vim in tmux was terrible; line by line scrolling was very jittery. Witching off font ligatures meant the GPU was able to handle rendering and now everything feels really smooth with multiple vim panes open.

Prior to moving back to terminal based editing I was very much in the same spot as the OP, using VSCode - I didn’t notice any form of latency with that setup.