You should check the literature to see if your experience is anything other than a psychological one. I can find no indication that a few percent minerals will offset the 97% sucrose in brown sugar. I agree the brown tastes better but a bit of color doesn't mean that there isn't something pure white and deadly (https://en.wikipedia.org/wiki/John_Yudkin) underneath.