iTerm2 with the new metal renderer (only added to the stable releases recently) is as fast as Terminal. But if you want to see what a truly fast terminal can be, give Kitty a try: https://sw.kovidgoyal.net/kitty/

As for OMZ, yep, it includes a lot of functionality out of the box which makes it a little slow. I personally use prezto because it's quicker. Give that a try.