I completely agree with you. For this exact reason I've made this plugin modular. Which means that you can include only parts that actually use, both from JS part (thanks to Grunt) and from CSS part (thanks to Sass).