It really doesn't have to be an all or nothing proposition though. Have you tried sticking mainly to low GI carbs, e.g., sweet potatoes, whole grain breads, etc? It takes a lot longer to digest these foods and, as a result, you get a smooth energy release instead of a huge insulin spike.

I've been into fitness, mostly around powerlifting and contact sports, for most of my life. Your body needs carbs; the problem is typically the form in which we consume them.