Maybe something's wrong with me, but I have to wash my hands for at least 20 seconds just to feel the soap residue wash away. If I don't, my hands have a filmy feeling afterwards.
I think it depends a lot on the water hardness from the sink you use, as well.
Hard water causes that, even (can be worse actually) if it's softened.