While I cannot find the reference, some researchers recorded full visible-range spectrum for a very large number of naturally occurring objects. Then they performed principle components analysis of these spectra and found that for naturally occurring reflection spectra, nearly all signal energy is recovered with a small number of independent components like three or four. In other words, the spectra of naturally occurring materials does not vary as widely as it could to exercise the full Hilbert space available. A downside of having a larger number of cones is of course that the percentage area available to each cone will be less, so a higher density of cells would be needed to maintain the resolution and color perception under dark will suffer more. The nature then has chosen a near optimal number of cone types for vision.
One issue I had discovered with the above study was that it was possibly cyclic. The database of spectra they used did not had too many samples of (different) objects with the same apparent colors since the latter was collected primarily to count the number of human-distinguishable colors occurring naturally in nature.
If the number of primary colors visible to humans were significantly larger, I can on the very least say that TVs and printers would have been significantly different than what they are, if at all they would have been practical.
The above however does not answer the question of the electromagnetic radiation frequency span sensed by the eyes. In other words, why is the visible range not wider than what it is. Again, some animals do have a wider range. Had the visible range been wider, it may have required a larger number of primary colors to cover the expanded Hilbert space. What a physicist once told me is that the range of frequencies that are visible to humans and animals correlates very well to the frequency range under which water is transparent. See [1]. This according to him offers a solid indication that life originated in water.
A few more interesting thoughts:
1. Figure out what determines the shape of the locus of the spectrum (monochromatic) colors on the color chart [2]. It is an interesting exercise to derive this locus from the measurements of the spectral responses of the three cone types. Note that the outer boundary of the color chart has a straight line bottom with purple to magenta in the middle. While the non-straight boundaries span monochromatic colors, the colors on the straight edge at the bottom cannot be monochromatic. Figure out why.
2. A question comes if the spectral responses of the cones could have been such that the above locus would have concavities. I am not sure of the answer, though am sure colors outside of the locus but within its convex hull would still have been visible.
[1] http://en.wikipedia.org/wiki/File:Absorption_spectrum_of_liq...
[2] http://en.wikipedia.org/wiki/File:Cie_Chart_with_sRGB_gamut_...