A big issue is that the human eye doesn't have R, G, and B sensors with well defined boundaries. Light frequencies that stimulate the cones receptors in the eyes cover a range of wavelengths from about 400nm to 700nm, see the first figure at Wikipedia's page on spectral sensitivity [1]. The three types of cones have largely overlapping sensitivity to light.

Light entering the eye can be thought of as having a range of wavelengths over the visible spectrum. A red source of light might be pure red, with a single wavelength right at the peak sensitivity for the red cone receptor of the human eye, but it might also be, and in most cases will be, a mixture of wavelengths some stronger some weaker falling across the visible spectrum that happen to stimulate the red cones more than the blue or green cones.

Visible light, made up of a collection of wavelengths, can be described as an intensity curve for each wavelength, that is, a real valued intensity function of wavelength. However, the brain is incapable of knowing the shape of this curve, it only knows how much each of the three types of cones are being stimulated. Two very different "colors" having wildly different distribution of wavelengths might be perceived as exactly the same by the human eye. Furthermore, the brain does a bunch of processing that changes the perceived color in strange ways, see for example, the Land effect [2], named after E. H. Land (creator of Polaroid cameras) because of his work on understanding it.

Finally, why isn't RGB enough? Each Pantone color, say Pantone Coated Warm Red C, does have an exact RGB (or CMYK) value, in this case #F9423A. The problem is that pigments in inks and pixels in cameras and the pixels in screens all generate or are sensitive to specific sets of wavelengths that won't exactly correspond with each other or with the sensitivity of human eyes. This is why we need color profiles and color spaces and all of the concomitant complications (especially so since the functions involved are not simple linear transforms).

A sophisticated enough sensing device could be constructed to exactly mimic the sensitivity of the human eye. It could report the apparent color of any object, say a fabric swatch, in three dimensions, say R, G, and B. Such a device could be used to compare a color in China with on in New York. It's just more convenient to use the ubiquitous color swatch strips produced by Pantone and match the colors by eye, and by now, one can simply load your offset printing press with ink matching Pantone Coated Warm Red C straight from the ink manufacturer.

[1] https://en.wikipedia.org/wiki/Spectral_sensitivity

[2] http://www.millenuvole.org/f/Fotografia/Per-quali-ragioni-ve...