I'm sorry if this comes off as uneducated, but why exactly would this be weakening your security?
When installing the Chrome plugin, Chrome clearly states which domain(s) it will allow the plugin to interact with.
I would see your point if this was for a banking site, or any other site where you would want to keep some information private, but I don't really see how this applies in this context.