In my experience, team building exercises just help you get more comfortable with your co-workers, regardless of whether or not each individual recognizes that it's happening.