Applied Machine Learning Is a Meritocracy
machinelearningmastery.com
machinelearningmastery.com
In short: article's claims sound fishy; your lived experience may vary.
Or are stories like this common and everywhere?
Very much so. Bad management is afraid of competence, because good performance cannot be argued away. I've seen this.
When I worked for a Fortune 500, credentials mattered. Possibly more than competence.
I'm working for a much smaller company now and it's significantly closer to a meritocracy (or a "who will generate the most obvious value - even if it means working 90 hours per week" ocracy).
It’s true till it’s not.
Worker: but you set the policy, it can be whatever you want it to
HR: ...
Saying “policy” must be a cover for a real reason that the company wants kept secret
In fact a version of the above story has happened in the last two companies I worked for.
As far as I can tell, even obsessing over credentials happens because its a proxy for merit[1], and we care about merit. Its often a lousy proxy, but our industry as a whole didn't care about merit, why would anyone care to look at a candidate's credentials?
[1] "Nobody gets fired for choosing IBM" -> "No recruiter gets fired for hiring a PhD"
Now I lead a team of analysts and junior data scientists at the University of Chicago.
That company is still making money hand over fist.
By the way, the point of the concept of meritocracy is that it depends on performance, not on some proxy variable.
A or B students could generally get the job done, but B students tend to be able to go deeper because of their greater theoretical horsepower. I could assign them tasks of greater difficulty and they will figure it out on their own and deliver.
If the organization only requires A-level talent, then let's be honest, there's no difference between A or B students. The majority of enterprises out there really only require A-level talent.
But if your organization has the kind of challenges that require B-level talent (a minority of organizations, to be sure), I think you might struggle with A-level talent. I faced this first hand when I learned that I couldn't scale because I was limited with the A-level talent that I had and was spending an inordinate amount of my time training and guiding them that I barely had time to do my work.
In general you are correct, but in some very specific situations, my experience has been there's value in identifying the right kind of talent that fits the problems you are trying to solve.
But there is sometimes more going on than meets the eye. If it's an A.I/M.L startup, the number of PhDs is a significant factor in fund raising. Being able to say "we have 6 PhDs on staff" will open doors.
Also besides a 'written job description' -- there are always unspoken 'tokens' for candidate's gender or, even, race, perhaps other attributes.
Studies on this type of subject are a taboo these days.
Promotions to different organizational levels in larger corps (with 20k employees or more, I would say) -- is like breaking through glass ceiling. You can see the other side, you can do the job on the other side -- but you cannot get through there...
Scott Adams, Dilbert author, had a personal experience about a promotion he was sharing. [1]
To me, it resonated, because it was closely matching my own at some point in my career.
Chess is a meritocracy. So is tennis and sales. Anyone can play. The best player wins.
Chess coaching is not. Better coaches might win more or get better gigs, but stuff other than coaching "merit" can play a big role.
But rather than getting stuck in a semantic argument... What the article means is that it's a relatively accessible field, on the way that software engineering is. The formal pathways are not as locked in.
A motivated and talented individual can bootstrap a career without formal education, qualification or approval.
All this is relative to, for example, civil engineering, law, medicine... There you'll need qualifications earned via established formal channels, gatekeepers and such.
I'm not sure you are having the correct definition of meritocracy in mind. A meritocracy is a system where the actors who provide the most benefit to the society, benefit the most. However, in chess, tennis, etc., there is no real benefit.
Originally it was coined as a satirical characterisation of a hypothetical stratified future society so obsessed with test performances in early years that all power became concentrated in the hands of adults who had demonstrated youthful aptitude and were thus assumed to not need to prove themselves any further thereafter since competition could only demoralise lesser minds; the author took the view that the "merit" of passing test thresholds would often be dependent on access to sufficiently expensive early years education and would resemble and often simply be a justification for upholding traditional social class divisions
As for tennis, I think it's fair to say that tennis fans consider Federer et al to provide them with considerable enjoyment, and far more so than less talented players whose games they don't watch, or indeed most people working in non-entertainment jobs who can only help a few people at once and/or find it difficult to prove how much better they are than other people in the same role
The problem here is that "meritocracy" reduces to "free market", where everybody is simply paid for the benefit they bring to others, and where only the few best win.
And my original point was that meritocracy wasn't even originally conceived as a "good thing", never mind a utopian distribution of wealth and power allocation according to some ideal standard of social utility. Other people have subsequently chosen to [re]define meritocracy as some sort of aspirational standard for allowing more talented people to succeed; people defining 'meritocracy' with respect to the propensity of that skill to generate revenue are making as valid a use of the term as anyone else.
Where X is [machine learning, tennis, chess].
So what is the take away, especially with applying this to all areas of our lives? We can do something to try to equalize the workplace, the chess tournament, or the tennis court. But what about equalizing the stage or the bar?
In terms of actual merit, as measured by actualities and not potentials, a healthy population is going to have more merit than an unhealthy population. If that feels weird to say, well, it's just a restatement of the why you want populations to be healthy and why you want to help unhealthy populations become healthy. It isn't because healthiness is somehow an abstract goodness disconnected from any real effect, it's precisely because it represents a real loss in capability.
Personally I don't like the idea of trying to equalize the results at the final measurement bar, like the chess tournament, because that just removes all incentive to improve actual health. We should strive to make people really better and healthier, not break our measurement tools for effectiveness and healthiness, thus guaranteeing a lack of ability to improve either.
I agree.
But how do we discuss reasons that are or aren't justified to be taken into account. For example you can break down potential blockers into temporary and permanent. Being sick or the death of a loved one can temporarily hurt your potential. Poor nutrition or a brain injury can permanently hurt your potential.
Do we judge people based on if they can overcome anything holding them back? Or do we wait for them to overcome it before we judge them? What if we are the gatekeepers of a method to overcome our limitations (such as the person deciding who gets to attend university)?
Would you mind characterizing briefly, or giving examples of, the successful projects you've taken on as a freelancer?
I've thought before about trying to go that road, but the fact is, for almost every ml project opportunity I've come across, I would have had to say, "well, I can try out some stuff, and it might prove extremely valuable, but just as likely as not my POC will not perform well enough to use in production".
There’s the nepotism portion (you need to get an interview first, before demonstrating merit, and nepotism bypasses a lot of that, and has the “they’re really good, just haven’t done much publicly/real world yet” backing).
Ignores that merit itself is not objective - just doing X doesn’t give an objective measure for how much merit X should get, and there’s a long history of evidence for non-merit/skill based bias in how much merit is “earned”.
Etc, etc
So far we've figured out that software development clearly isn't a meritocracy.
Speaking personally, I grew up in an incredibly deprived part of my nation, I have little in the way of formal education and I definitely do not have connections. However I have achieved incredible social mobility due to technology. Why? because there is _some_ level of meritocracy going on even if it's not entirely widespread through every single hiring manager or company and I have a deep unyielding passion for this field that drives me to continually educate myself which despite my poor connections and education is looked at favourably.
I still think of meritocracy as a noble goal, and I feel that merit is certainly definable if that's the main issue (IE; points based on time spent contributing to open source and things like invitations to speaking engagements and the like).
I genuinely do not buy the argument that having "merit" in other areas infers that you have license to be a toxic asshole; because at (least in my mind) part of merit is being able to work with others. You don't have "merit" if you can't communicate effectively.
In fact, there is an old saying about that: "Lots of MIPS but no I/O" which would make for a shitty CPU. Why would it make for a non-shitty developer?
There of course are still standards and lots of disagreement on whether they are merit based (e.g. whiteboarding interviews) and evidence that women and other minorities are systematically excluded in the whole, which is clearly not merit based.
Further whilst I think many of us could define what merit means for any particular job or find consensus in a group making that decision. I still expect there would be large differences between definitions and methods of selection.
Claiming that a profession is meritocratic, though, strikes me as a harmful framing. From my vantage I don't see a) what an objective notion of merit would be and b) that all people who have the capability to contribute is strictly equal to the number of good jobs that are currently available.
I can definitely see when someone is already doing something useful well, but not if they might do that later or if the notion of useful will change.
Of course what is considered meritorious or an achievement is almost entirely dependent on the stakeholders in power above you and rarely has any purely objective measure, and any objective measures are almost always project dependent, making it all a highly political game to figure out what's important to whatever system is judging you.
However, this just boils down to people's preference, are you a bottom up learner, or not. Both ways have yielded success for many different people. So what is the text actually saying?
Some very good people investigating it theoretically. But the best young math professors that I’m personally friends with actually do have something of the “elitist math attitude” and will tell me things like they think the ML field belongs in the 1600s while they successfully work in their obscure area of algebra applied to mathematical physics or something
And I acknowledge that we often don’t do a good job of this; but surely that doesn’t mean we want to move away from merit based decisions.
1. How meritocratic are the decisions we make today?
2. Do we want more meritocracy, or less?
I think there's specific ways we fail to make meritocratic decisions (credentialism, "culture fit", big company politics). But in my experience these things are the exception, not the rule. We don't start by trying to figure out if someone is a good culture fit, then assess their technical skills afterwards. We assess technical skills, then if we like what we see we check culture fit.
I don't know about the GGP post, but I've certainly talked to people who claim to reject (2) - saying that the whole idea of meritocracy is flawed. But I'm always confused by that argument. Would you accept a rubbish pull request in a personal project? Would you hire someone with no technical skills into a technical role in your company? If the answer is no, then you're on team meritocracy-is-good.
"Not very" based on my experiences.
> 2. Do we want more meritocracy, or less?
Ideally you want more meritocracy -but- that relies on a level playing field which is much, much, much harder to implement and would involve people paying taxes, etc.
> Would you accept a rubbish pull request in a personal project?
Honestly, I might, if it was something I was never going to get around to and it scratched >50% of an itch. Or if I could see it had potential for later work. Or if it was from someone I wanted to have contact with.
> Would you hire someone with no technical skills into a technical role in your company?
If you add "relevant" after "no", you'd be surprised how often this happens.
https://www.theguardian.com/politics/2001/jun/29/comment
"It is good sense to appoint individual people to jobs on their merit. It is the opposite when those who are judged to have merit of a particular kind harden into a new social class without room in it for others."
I think you can see inklings of this in support for Trump, Brexit and distrust of elites the term for the new 'meritorious' social class.
Further merit in a job should extend well beyond technical skills.
Reading that article it looks like the term 'meritocracy' was coined to refer to the former, but we didn't have a word for the latter and so the word has been co-opted and muddied. "Meritocracy is a discredited concept" could thus mean any combination of these statements:
1. We don't / can't currently hire based on collage degrees
2. We don't / can't currently hire based on relevant ability
3. We shouldn't try to hire based on collage degrees
4. We shouldn't try to hire based on relevant ability
People in this extended comment thread seem to be arguing assuming that these stances are interchangeable, or arguing assuming that there is consensus about which of these points we're arguing and they know what it is.
Personally, I believe that we should try our best to hire based on relevant ability. Because I want the most qualified people as coworkers and creators of the products and services I consume. I welcome debate of the idea, but arguing about meanings badly is pointless.
Slatestarcodex wrote a great piece about this whole thing a few years ago. I highly recommend having a read: https://slatestarcodex.com/2017/07/24/targeting-meritocracy/
> The intuition behind meritocracy is this: if your life depends on a difficult surgery, would you prefer the hospital hire a surgeon who aced medical school, or a surgeon who had to complete remedial training to barely scrape by with a C-? If you prefer the former, you’re a meritocrat with respect to surgeons. Generalize a little, and you have the argument for being a meritocrat everywhere else.
Take for example your quote from Slatestarcodex. That's purely about qualifications. Surgeon A might be an alcoholic and Surgeon B might have an incredible success rate. Are their qualifications relevant then?
Anyway the point of the article and book isn't about merit as just qualifications but about the detrimental impact of rewarding only merit on the stratification of society. Changing the criteria for merit just alters the stratification.
Often, things are named by those that do them the greatest injustices.
I don't mean that to be inflammatory at all by the way; but there are people in the world who believe the circumstances of their place in society plays a significant role in how you should weigh them and their contributions.
So, I wouldn't jump to "discredited" just because there's one blog post from a surprisingly influential person. I would instead say "contested".
"Affirmative action" is the PC term for what he described, and is pretty broadly accepted...
More exactly, it is the psychological trait "Belief in a Just World" (BJW.)
One doesn't have to look far in our actual civilization to see innocent victims, injustice, but that still doesn't penetrate the existential strata of the universe eventually righting all wrongs.
Here's a good analysis of the psychological studies: https://www.youtube.com/watch?v=aQ0lR23T7FA
If you don't have time to watch the video, the most significant points are:
1) Believing in a Just World / Meritocracy, is an evolved emotional buffer to the chaos and reality of ever-present injustice.
2) BJW trait correlates with higher measures of happyness after traumatic events like natural disasters - and seems to be a trait that remains relatively static among persons regardless of the tradgedies they suffer or windfall they receive.
Am also in the middle of training a group of future data scientists. The company is having issues hiring, and believe it is easier to train SME's in data science than the reverse.
https://www.greaterwrong.com/posts/dLbkrPu5STNCBLRjr/applaus...