And social life in colleges is (sadly) dominated by drinking at the college bar, drinking in clubs, and dinners in the dining hall, where everyone seems to sit according to some unofficial hierarchy (with being in the top sports team in a popular sport seemingly correlated with being near the top of that hierarchy). Inter-college and Oxford vs Cambridge competitions in all kinds of sports (but primarily rowing) are a massive part of college life. The university rewards athletes with awards of blues and half-blues, drawings on college walls celebrating sports victories, prominent mentions in college publications etc. Academics, on the other hand, are not given nearly as much attention - you would almost never know how good someone is doing in their studies, especially since they are probably studying a completely different subject to you.
It would be common to go and support your college sports team in, say, rowing; whereas nobody really cared so much about academic competition between colleges. Those academic victories were never really mentioned or celebrated.