AFAIK this has only been a real problem since Yosemite. I noticed it when running the GM Candidates but never saw it in Mavericks.
Either way, the weird thing is that if you go to the Window menu and hit Task Manager... you can monitor extension CPU usage, etc.
I never had a problem until recently, and yet now I'm seeing red in the Activity Manager due to Chrome. It's bizarre.
I only use Chrome if I need to use a site with Flash and I use the "Open Page With" item in Safari's Develop menu to open the appropriate page. Then, I quit once I'm done with it.