No, they would know by the angle of your eyes if you were staring that low.

Looking intensely at someone's lips is at times the prelude to a kiss. I think this ambiguity makes them uncomfortable.