There are women's colleges that exclude men, and there are many liberal arts college that given men a preference to avoid having the sex ratio become too lopsided. Congress have never voted to ban sex preferences in college admissions.
This is a constitutional issue, not a law passed by Congress.