I see the issue with false positive, but I don't get the privacy argument here. Scanning your face vs your passport is not any different from privacy perspective.
Imagine this:
"Years from now, the party/person in power goes completely nuts, and starts rounding people up in train cars to ship off to a final solution facility... would you prefer to try to sneak out of the country when: A. only your passport/papers identify you, or B. your face can be recognized from any video camera in the country..."
Just a few weeks ago there was a thread on this site discussing how IBM census taking machines made it all possible to identify the undesirables. Now it's trivial.