That's the thing, you could go either way. I am not sure I can answer the question "what would a resonable person click?".
The same is true with, say, buses. See an image of a delivery van? Bus! It asks you select all cars and you see no car but a vague pixel blob that someone stupid would identify as a car? Car!
One of the few things that this doesn't work with is stairs, because the side of stairs being stairs or not is something apparently no one can agree on.
For example, hesitation/confusion patterns in CAPTCHAs are different between humans and bots and those can actually be used to validate humans