> How Candidate Order in Prompt Affects LLMs Hiring Decisions
Brb, changing my name to Aaron Aandersen.
In the Monty Hall problem there is added information in the second round from the informed choice (removing the empty box).
In this problem we don't have the same two-stage process with new information. If the previous process was fair then we know the remaining candidate was better than the eliminated male (and female) candidates. We also know the remaining female candidate was better than the eliminated male (and female) candidates.
So the size of the initial pools does not tell us anything about the relative result of evaluating these two candidates. Most people would choose the candidate from the smaller pool though, using an analogue of the Gambler's Fallacy.
Maybe my code is buggy.
I tried an implementation with the values being integers between 1 and 100, and I found stats close enough to yours (~51% for 10 elements, ~64% for 100 elements).
When using floating point or enforcing distinct integer values, I get 50%.
My probs & stats classes are far away, but I guess it makes sense that the more elements you have, the higher the probability of collisions. And then, if you naively just take the first 2 elements and the female candidate is one of those, the higher the probability that it's because her value is the highest and distinct. Is that a sampling bias, or a selection bias ? I don't remember...