Those were two examples, but they're not exhaustive. Traffic direction can (and often does) happen without someone blocking the vehicle.
In the case of an illegitimate request, a human driver might question the request and take a different action instead.
Are you being directed to run a red light? If you judge that someone is maliciously asking you to do so, you probably will choose to remain still instead.
Are you being directed to stop? If you judge that someone is maliciously asking you to do so, you may choose to turn around, reverse, turn into a parking lot or otherwise remove your vehicle from the situation. You don't have to run people over to refuse a command to stop.
And if you were completely blocked from moving, you'd probably call the police.
My point is that "always follow hand signals" is trivially problematic. I would expect that if self-driving cars followed any hand signal they saw, you'd see kids doing it for the lulz on the sidewalk in front of their school. ..."traffic jam challenge". After all, someone is less likely to act maliciously towards a human than they are a machine.