I don't experience an internal monologue and don't think of any words at all when looking at colors, yet I have a very good ability to differentiate between subtle hue variants (based on tests I have taken that test exactly this).

If it's true that language affects ability to differentiate hues (I am somewhat skeptical, but certainly open to the possibility), then it seems to me that maybe language just muddies our natural ability. In other words, thinking about the word "blue" might cause you to artificially group colors together that you wouldn't otherwise group.