An article linked in the OP shows that default Terminal.app is more responsive than Alacritty or iTerm.
And yes, scrolling with j/k for longer periods may be an anti-pattern in Vim and yada yada... I scroll in Vim, and in Alacritty its smooth, in Terminal.app its sluggish.
That said, there are plans[1] to reduce Alacritty's input latency. Though, I personally use it as a daily driver and have never felt that there was a noticeable input lag.
Once that lands, Alacritty will have similar latency to Terminal.app _and also have_ a 60 Hz refresh rate (the "smooth" feeling), low CPU usage, and much higher throughput.