The later we meet someone in a sequence, the more negatively we describe them
suchscience.org
suchscience.org
First because you set the benchmark, and your outstanding qualities become "requirements" for the candidates that follow to meet that benchmark.
Last because of recency bias, so whatever qualities you have are better recalled by interviewers.
Everyone else becomes somewhat forgettable.
(I would guess they mentioned a study about it, but it's been 10 years and I don't have the reference handy.)
In my experience as an interviewer, everyone in the middle does get somewhat mixed up together, especially when I had less than 5 minutes to scribble notes and reset between candidates.
But I would modify "first" and "last" to "towards the start" and "towards the end" e.g., I will (subconsciously) more easily benchmark candidates against a very strong "second interview of the day" than against a lackluster "very first candidate", if that makes sense.
Said differently, whoever is the first candidate that hits it out of the park becomes the benchmark. And whoever happens to be the best relatively strong candidate is more easily recalled than other relatively strong candidates.
Thank you for coming to my TED talk
N = total applicants
1. Interview N/10, decline no matter what
2. Hire the next person who's better than everyone seen so far
I think the magic number is 1/e of the applicant pool (e = Napier's constant, 2.7...)
The main problem here indicated by OP isn't explore/exploit, it's that the ordering of candidates seemingly distorts their rating of them.
But above all, do well and don't overthink this stuff as there's no scientific rigor. It doesn't help to be first or last and suck at it.
One of the issues we had interviewing very young candidates (like fresh out of college) was how they all looked awkward, even in their outfit, and few had any kind of confidence during the 15~20 min we spent with them, with some saying really weird things (stuff like "I'm really good at the internet"). But obviously non of that matters long term, we assumed they'd probably all fit in fine once in. Putting some distance and picking up the good and bad stuff from the transcript helped a lot to get past the weird impressions, including the order we saw them and how tired we were when we saw them. To some point.
https://onlinelibrary.wiley.com/doi/abs/10.1002/ejsp.52
>The durability of anchoring effects
>Recent research suggests that judgmental anchoring is mediated by a selective increase in the accessibility of knowledge about the judgmental target. Anchoring thus constitutes one instance of the judgmental effects of increased knowledge accessibility. Such knowledge accessibility effects have repeatedly been demonstrated to be fairly durable, which suggests that the effects of judgmental anchoring may also persist over time. Consistent with this assumption, three experiments demonstrate that judgmental anchors influence judgment even if they were present one week before the critical judgment is made. In fact, the magnitude of anchoring was found to remain undiminished over this period of time.
I didn't get access to the full text, but had a look at other papers from the same researcher [0] on what kind of methodology they use.
In the case of recruiting, I think the main factor when moving the decision further down the line is the change in information ("a selective increase in the accessibility of knowledge about the judgmental target"), in two specific ways:
- we actually remember less about the subject, for better or worse. A candidate might have had a weird look, and the notes are probably impacted by that bias, but we can look back at their coding test without that impression and come out with a slightly different conclusion.
- we get to compare to other subjects in a different order. In particular, that helps catching weird expectations. For instance if every candidates has been falling through the same trap, it helps give them a pass and assume the question was at fault. If we had to do that in real time, only the last few ones would get a kinder judgement.
[0] https://www.researchgate.net/publication/11394075_The_Mallea...
*shudders
But I would not want such things to be taken very seriously unless you trying to fill a very narrowly defined post because it is all to easy to create a test that a good candidate would fail.
In particular very simple tests (like an API interface, or reversing a string etc.) done in any language they feel comfortable is are usually a trove of info about the candidate. The result doesn't really matter, it doesn't need to run, it doesn't need to be complete, as long as you got to hear a lot about how the candidate thinks, how he moves through the problem, and how much they can write something basic, what they're confident in and what they're not used to do etc.
I would compare it to reviewing one's code a few hours later. We're still in a similar mindset, but there's a bit more distance, and we also catch the bits that don't make sense when reading back afterwards. That works even better when exchanging notes afterwards.
In particular we had people who's surface personality were completely different from what we perceived during the interview. Not in any way that made it hard to work with them, but moving from university to a corporate environment was just enough of a gap to change their behavior in significant ways. I think hiring fresh graduates is way harder in that respect, and we were happy to have some flexibility in the work culture. One of the guy moved from a super rural area to be thrown into the megalopolis, and it was a real journey, we had the funniest late to work excuses ("couldn't find my bike cause I parked it near the neighbor's house and he moved it in their garage thinking it was his son's" -- later found his house had a bike parking behind the building)
https://www.jcr-admin.org/preprints/chernev-preprint.pdf
>Semantic Anchoring in Sequential Evaluations of Vices and Virtues
>How do people evaluate sequentially presented items? Prior research suggests that sequential evaluations are subject to anchoring biases, such that the values of subsequently evaluated alternatives are assimilated toward the initially considered option. The present research argues, however, that sequential valuations often lead to contrast rather than assimilation effects, whereby values of the subsequently estimated alternatives are distanced from the initially evaluated option. These contrast effects are attributed to semantic anchoring, which stems from evaluating conceptually related options classified into opposing categories (e.g., vices and virtues).
That situation obviously doesn’t apply if you’re interviewing many candidates on the same day and comparing them to each other. As a result to optimal strategy for the secretary problem also doesn’t apply.
Do you have any explanation for that ?
Or maybe it's also more about calibration over time.
Most of our social technology is about not being shitty to one another at scale (or maybe "in massive sequence"), so this seems aligned with my understanding of the world. The work of progress is to "not be shitty" at increasing scale (of population, idea complexity, levels of abstraction), and we build institutions that mostly try to do that. Though I think digital has kinda failed on that mission lately, which is another conversation
Seems kinda nice and adaptive and optimistic even. Though yes, downsides in a society that lives at scale, if not mitigated by process or social/digital tech.
Put it another way: we're shallow, primitive, creatures.
What you're calling "primitive" (perhaps with judgement) is part of the system that perpetuates our collective socio-biological process in poorly understood ways.
For all we know, liking ppl until we're at scale is highly adaptive (in an information-theoretic sense) for sociality as a whole, and not some broken primitivity :)
Put another way, we typically tend to sort first for assholes, then maybe sort for other things later.
The idea of "the tyranny of the database" was where I first encountered this:
> At a higher, more semantic level, a subtle distortion in how we perceive reality took hold: things that were hard to represent in databases became alternately devalued and fetishized. [...] Once in awhile a technical counter-current would take hold and try to push back on the tyranny of the database, but the general trend held firm: if it does not fit in the database, it does not exist.
> You may not think you know this world of databases, but you live in it. [...] Every time a customer service assistant shrugs and says “computer says no” or an organization acts in crazy, inflexible ways, odds-are there’s a database underneath which has a limited, rigid view of reality and it’s simply too expensive to fix the software to make the organization more intelligent.
https://medium.com/humanizing-the-singularity/by-the-end-of-....
I don't have access to the study, but I'd be curious why they chose to count positive words as opposed to just asking people to rate on a numerical scale? My impression is that sentiment scoring via bag-of-words is not a particularly robust method, especially in 2024. It also sounds like they didn't normalize by description length, so outcome could just as easily be because people's responses got shorter as time went on due to fatigue?
(also, this is a nit with the article rather than the study, but given the methodology I think it is important to distinguish between becoming less positive and becoming more negative, and in this case I would not describe using fewer positive words as "became significantly more negative")
The first round is a basic filter. By the time you get to the final round you're only thinking about a dozen people.
Full study is here: https://osf.io/s2zv8/download/?format=pdf
https://en.wikipedia.org/wiki/Anchoring_effect
More specifically, sequential anchoring:
>Consider the following scenario: A reviewer evaluates many unqualified applicants for a university program successively, and the next applicant to be reviewed is an average (borderline admit-reject) applicant. Because the evaluator is influenced, or anchored, by recently made decisions, this borderline applicant might be admitted to the program. On the other hand, when an evaluator is anchored by having reviewed many qualified applicants, the same or a similarly borderline applicant might be rejected (Figure 1). In this scenario, individual fairness, stating that individuals with similar characteristics should be treated similarly [8], is impaired, and wrong or inconsistent decisions can have a consequential impact.
Source: https://dl.acm.org/doi/fullHtml/10.1145/3491102.3517443
This kinda means it's a waste of the candidate's time unless they're _also_ just interviewing for practice.
1. Tightwads - spend as little as possible (maximize savings)
2. Spendthrifts - spend everything they have (maximize pleasure)
3. Optimizers - who want the best deal possible (money for value)
4. Satisfiers - who set a bar then pick the first option that meets it (minimize decision time)
You see these behaviors in all sorts of decision-making processes as well, so if the hiring manager is a Satisfier type and the first person who walks through the door is great, they'll likely hire them.
If the hiring manager is a tightwad or an optimizer though they'll almost certainly force many candidate through to collect data before making a decision and that first candidate has little chance unless they're truly exceptional.
A person that is aware enough to know they are not calibrated should also not make the mistake of attributing higher/lower odds to something that is mutually exclusive, assuming that the interviews are not systematically set up in a way to put worse qualified candidates first.
I would think that there are two forces at play here:
1) the ones who come first are not compared against any of the latter ones, so make a better impression
2) the ones who come last are still in memory, especially if results are all tabulated in the end.
So, if the above is true, and there is no guarantee that it is, I would have expected an inverted bell curve.
The examples they give: Job interviews, dating, auditions, are all cases of search for the best out of a set of possibilities. The earliest candidates have the initial advantage of there being few in the set, or none, for the first candidate. Since the purpose is to compare the entire set, by definition, the first one is the best you've seen. Even if they objectively aren't great, the rating at that time will tend to be more positive. That isn't a bias, it's a best-effort judgement in the face of incomplete information. You'd need a crystal ball not to do this.
What they're measuring is immediate impressions, while what they should be measuring is decisions made after the search concludes. I've certainly had the experience of interviewing a candidate for a job and telling myself "yeah, might be ok", then a later candidate comes by and I adjust that to "no, this one is much better". If they want to show that being first or early in such a sequence makes it more likely someone will get picked, they've failed to do so with this protocol.
I'm a psychologist, and though HN usually tends to be tough on psychology as a science, I'm amazed that this paper is being discussed so earnestly. Social science in particular should require a bit more skepticism.
For what it's worth, my preference during my time as a working actor was to audition first thing after lunch. I figured (based largely on my own experiences as an auditor) that the first people in the day would be forgotten / calibrated against, and those later in the morning would be victims of fatigue, whereas directly after lunch I'd have the best chance to make the best impression on fresh auditors. I had objectively excellent rates of success, so figured that method was good enough. <shrug>
I don't think that generalizes to more extensive interview processes. Auditions are weird: you get a "hello", then five minutes (max) to do two monologues, or one monologue and a song. It's basically a cattle-call, and very hard to stand out. Sometimes the "hello" is actually the most important thing, sometimes what you're wearing is what makes the best impression. As an actor you have essentially zero control, and yet that gives you utter and complete freedom.
Still: first thing after lunch. Works every time, except when it doesn't.
Alternative interpretation:
Paid participants who most likely just want to do the study and get paid get more and more cranky the more pointless words they have to write about unknown people’s Facebook profiles.
How well this actually reflects phenomenon in the real world is actually unknown.
This reminds me a lot of https://datacolada.org/98 and if I think about it, the data would be trivial to fake.
See:
https://www.cambridge.org/core/journals/judgment-and-decisio...
>A number of studies have shown the robustness of the anchoring effect. Anchors can influence judgment even after weeks or months (Reference MussweilerMussweiler, 2001; Reference Yoon, Fong and DimokaYoon & Fong, 2019). The anchoring effect is present even in experts in the judgmental domain (Reference Englich and MussweilerEnglich & Mussweiler, 2001; Reference Englich, Mussweiler and StrackEnglich et al., 2006). An anchor can influence subsequent judgment even if it is clearly implausible (Reference Strack and MussweilerStrack & Mussweiler, 1997) or when it is compared to a different object (Reference Frederick and MochonFrederick & Mochon, 2012; Reference Mochon and FrederickMochon & Frederick, 2013).
Here's hoping someone with a backhoe doesn't check for fiber conduits.
I'm not trusting any result like this unless it comes from a meta-analysis really. I have heard of this general idea before, that the first and last to interview are more likely to get a job, but the plausibility of the finding doesn't make it any less suspect.
Donald Trump tends to like the last person he spoke to best, but probably because that's the person who comes to mind when asked, and anyway he doesn't really care one way or the other.
Unless the topic of conversation was himself: Donald Trump, in which case he cares a lot, will remember details, and will like whichever person said the nicest things about him, regardless of sequencing.
>AI-Moderated Decision-Making: Capturing and Balancing Anchoring Bias in Sequential Decision Tasks
The latter you go, the more desperate the group will become. Fatigue and whatnot.
If you are their first after the lunch however, that would again be a very good spot.
This seems to be the opposite of "recency bias", which is interesting - if true.
Spoiler: I’ve not been able to notice any difference.
Makes me wonder if the function wouldn't end up being like hypergamy.
Makes me wonder (as a follow up) if turns up to be like hypergamy, then there is a link between human economics and sexual behavioral drivers.
EG - I recently watched Peter Pan for the first time, as an adult. Peter is basically a horrible person who deludes and kidnaps children, puts them in incredibly dangerous situations, doesn't care about people other than to make himself look good (eg, he fought the pirates who had kidnapped the Indian princess, then he got so distracted w himself she nearly drowned in the tide). On the other hand, we're told that Hook and the pirates are the bad guys but really Hook is just responding to the fact that Peter had maimed him and fed his hand to a crocodile. There's no indication of anything bad Hook has done prior or since, other than trying to "get" Peter for having done that.
The protagonist of Peter Pan is actually the dad who was trying to protect the kids from nonsense but even he got derailed by the wife, therefore leaving the kids vulnerable to all that has happened to them.
But if you ask most people who are familiar with the story, they'll react to the superficial presentation that Pan is whimsical and aspirational for children while the Dad represents the dull adult world to be escaped from, or something like that.
The crux is that the narrative form suggests how we should feel about whom even if that makes no sense. I think this is what the article here is talking about as well in a different manifestation.
Hook is a pirate.
By definition, that's a murdering, raping thief, whos main income is boarding honest ships, slaughtering their crew, etc.
Piracy is/was punishable by death.
The story is old, but at the time written, anyone would be a hero for maiming, or killing a pirate.
I’m going to contest this. Peter Pan first appeared in 1902[1], by which time the romanticization of pirates was well under way. The Pirates of Penzance, for example, debuted in 1879[2].
This doesn't mean they're good people, and that 120+ years ago they wouldn't be put to death. Anyone killing a murdering, scummy gangster would be a hero.
Just because someone writes a nice fictional comedy story about a scum of the Earth group of pirates, doesn't mean that all pirates are OK. Ask the villagers that get raped, murdered, and killed by pirates what they think.
And yes, that's in parts of the world today too.
It seems like you're on Pan's side, which I think is in line with the point the comment you're replying to makes.
there is no such thing as people with identical profiles
Especially in current time where justice is understaffed and judges have to work long hours...
This is quite scary in my opinion.