I discovered this issue 2-3 years ago. On slightly older machines, there was a palpable startup time. My fix was going through OhMyZSH and stripping away all the parts that I felt unnecessary (I call this my "leanZSH" and its considerably lighter version of OMZ.) It doesn't track upstream, and I manually update the plugin directory once in a while. Surprisingly OhmyZSH is pretty modular and doesn't break easily.

[Not the best hackjob out there but here it is:

https://github.com/gradientwolf/leanzsh

If you want to update it just copy over the latest `plugin/` folder from OMZ repo. You can get rid of all the plugins you dont want, as well as the themes. It somehow works]