Thinking back, I recall feeling uncomfortable commenting on or diverting the conversation towards anything that was unrelated to the prescribed topic of the meetup. It felt rude to divert time away from the topic that everyone there had specifically chosen to allocate their time to, otherwise they wouldn't be there in the first place.
With places like school, even college, there's still a general feeling that you have to be there, and so diversions from the topic at hand are more welcome.
To put it another way, interest-based groups seem to be about the interest first and the people second. The people there are compartmentalized away as being related to the specific topic, and not generalized friends. In this way the group lacks that crucial idleness factor that others here have mentioned, since everyone is there with a purpose to fulfill that they don't want to distract others from.