If you pin the extension in the toolbar, there's an option to "disable" the extension. The floating popup goes away and using the extension to summarize still works.