Well, one of the reasons for washing and drying ones hands is to reduce the amount of bacteria exposure. I don't have the same expectation when shaking a dozen peoples' hands at a party.
I don’t think anyone can expect that washing your hands will eliminate all bacteria- including those in the air.
That said, I still hate hand dryers and still refuse to use them.