Consider that just after the cone cells, there are other cells doing some computation / aggregate of cone signals. Don't forget that color is a brain construct.

For those reasons (and others), there's often a strong disconnect between stimuli and perception, which means there's no such thing as a perceptual uniform color space.