Sorry, what I say was probably not clear enough. I agree with you, eye contact is a way to communicate something. It may be defiance (that's what I meant by "it might be linked"), but it could also be seduction, fear, friendly etc. And we can accept most looks much longer than animals.
I just had a strange experience in Paris subway : a complete stranger on the platform (seemed tough) and I on the train did catch each other looks for a good few seconds and ended up smiling at each other. It was not the first time that I put myself in a friendly mood and got friendly looks back at me.
Pretty unusual, as shown in this (french) cartoon
http://www.dailymotion.com/video/x7fp4s_18-baston-de-regard_...
(you don't need to understand French to get it)