oh-my-zsh is also horribly slow out of box, spend some time manually configuring zsh and observe your terminals open in fractional seconds instead of multiple seconds.
As for OMZ, yep, it includes a lot of functionality out of the box which makes it a little slow. I personally use prezto because it's quicker. Give that a try.
I'm familiar enough with the terminal that I generally type by rote, I don't even need to look at the terminal while typing for most common tasks.
It's basically exactly what I had before except I can't click on links in my terminal.
iTerm2 is nice but for my usage it does too much. If there were a lite version that cut a lot of the more niche features and focused solely on speed and responsiveness I might consider it.