I recall the experience of a blind adult getting surgery that allowed for vision, and describing the process of learning visual information gave some idea of what infants experience; first it is all brightness, or darkness, then very vague shapes and primary colors, eventually honing in on discrete objects and hues.

It makes more sense that, without any context, our brain can't just "add" a color to its repertoire, but must interpret some data within the context of existing 'thought'. Some things may be literally unimaginable without a framework of existing memories and experiences, or only possible through some recursive process "Oh so 'a' is 'like' xyz, that makes sense".