While writing your own feature detection is still probably lightweight (although not by much if you customize modernizr correctly), you're not going to get all the edgecases Modernizr will unless you spend A LOT of time on it.
If you're using the majority of Modernizr tests it makes sense to use the full library, but for the most part me and the other devs I've talked to usually drop the whole library in for ease of maintenance, but mainly use it for one feature alone: touchscreen detection.
Lately I've tried putting some of my HTML on a diet by replacing my need for Modernizr for the purpose of touschreen detection with this snippet:
if(('ontouchstart' in window)||(navigator.msMaxTouchPoints>0)){
// code for touchscreens here…
}
Suppose Modernizr were a book, and each feature Modernizr tests for was a recipe. What I'm suggesting is this: Instead of going to the library and checking out the entire recipe book just to refer to the icing recipe, just copy out the list of ingredients and leave the book on the shelf. you're customizing the recipe anyway, so if you just add their list of ingredients to the steps you're writing for your own customized icing it's going to be easier in the future than if you write your recipe steps with a little note that says: 'refer to ingredients on page 24 of Modernizr Cookbook'. I also think buying a personal copy of the book for the same purpose is overkill - it makes more sense just to list the specific ingredients you want to reference directly inside the new recipe you are writing and keep it all on the same page, plus not have to worry about where that recipe book is every time you want to whip up a new batch of icing.