The eye is receptive to three colors, that's it.
We cannot tell the difference between a color which is a mixture of two (or more) frequencies of light, and a pure color, whereas we can tell chords from notes. (In spite of notes having harmonics).
We may be able to point at a red object and call it red. But there are are so many hues of red that this is about as accurate as being able to identify which octave a note is in. When you think that two objects are about the same hue of red, and the put them side by side, you generally find that they are totally different. Color also changes with lighting. A uniformly colored surface does not appear to be the same color if it is not uniformly illuminated, or does not uniformly scatter light in all directions.
When it comes to sound, we may be poor at identifying a pitch, but it seems we are fairly good at identifying EQ curves. Firstly, we can recognize people by their voices, which are the result of a tone's profile being shaped by the vocal tract. In relation to this, we can tell an AAAAH from an IIII, also, regardless of the speaker's pitch: whether the speaker is a man, woman or child. Or even whether the vowel is being whispered. Speakers of languages that have certain vowels that are very near to each other can distinguish those vowels, like some higher "a" versus a slightly lower "a".