I think it depends on the culture of the group. If the group isn't cohesive enough to have a culture, you're doomed. I've been to conferences where if you didn't know someone important, you were just another mark.

My experience at YAPC, now "The Perl Conference" has been different. The culture there is very open. I made it one time, I'm no luminary in the Perl community, but Larry Wall invited me to lunch. The other speakers were also very open and approachable.