I feel this is wrong way to optimize: First, bootstrap gzipped is <100KB after removing font. Check any reasonable site from youtube to amazon, they download ~ 1MB gzipped data on load and do just fine. Second, if you are really concerned, you can just remove a lot of components from bootstrap to reduce its size. I do it for my project where i don't need for example the glyphicon, carousel, etc.

When using a framework, you want it to just work on all platforms. When I look at bootstrap issues, I see so many issues of the form X does not work on Y and then it gets fixed. These little kinks are hard to get right and I am not sure its worth reinventing the wheel unless there is a clear benefit other than size.