I had the same problem -- after throwing together a few open source front-end libraries, and struggling with each to get webpack, babel, karma, flow, etc. all working seamlessly together, I put together https://github.com/krakenjs/grumbler and now just fork this every time I want to start something new.
Being able to just start writing code without spending forever setting up the perfect environment is hugely liberating. I'd encourage everyone to find good boilerplates for what they want to do (or put together their own if nothing else fits)