Dont forget that when colleges didnt exist for everyone, say 200 years ago, most people were farmers.
People are selective when it comes to timeframe, but it's just a mean regression - some periods reach peak, some get back lower, but all in all, we're trending up.