This econtalk provides some nuance for your point:
http://www.econtalk.org/archives/2014/04/bryan_caplan_on.htm...
There are many reasons to believe that college is more about signaling than knowledge. While admission is an important part of the signal (certainly at the high-end), completing college is probably an important signal in the mid-range and below. Completing college is probably associated with work-ethic, the ability to follow rules successfully, etc.