Certifications create wrong incentives for engineers and recruiters | Hacker News Reader

Certifications create wrong incentives for engineers and recruiters

interviewing.io

85 pointsleeny79 comments

Author/OP here. Just heard an interesting and relevant tidbit from someone who read the post. Apparently a while ago, Google fed resumes of employees into an ML model and correlated several metrics to the success of a specific engineer at Google. There was a lot of interesting data gathered from that but one that stood out was exactly that the number of certifications listed was inversely proportional to the engineer's success (measured as career progress).

I think that’s largely because people who get certificates often do so because they didn’t have formal (or close to formal, e.g. highly motivated dropouts) education in the field they are getting a certificate.

(also - job churn, and/or they may be slightly less autodidactic)

Exactly! You get certifications to prove you can do a job (in theory); the most successful people never have to prove they can do a job.

Go look at all of Carmack's certifications, or Torvalds, etc.

except, bullying people over perceived "lack of" whatever is first stop for abusive job shops. The problems of "the most successful people" are not the problems of a million+ twenty-somethings trying to work for a living. Please recall that in some nations right now, the majority of the population is under 25. Literate people are being setup to be cheap IT labor right now, fair or not.

from my experiencing interviewing, people get and list certificates not for that reason but because they have less awareness that those certificate are considered BS and therefore listing them indicates how you don't recognize BS.

I'm a "highly-motivated dropout" and I accumulated a few certs in a previous life because my employer paid for it and it was for areas in which I already had some knowledge (mid-level CompTIA stuff, Agile, AWS). I'd typically buy the book, read through it in week or two and highlight it to shore up my weaknesses, take notes, then do the test. I never once failed via this method. As it turns out, however, this is a very niche approach. The vast majority of my peers looking to get such certs would go to a "bootcamp" and spend months memorizing test banks compiled by third parties.

There's a gulf of difference between somebody that gets a cert to validate/enhance their knowledge and somebody that does so to get a piece of paper. The amount of people that fall into the latter category seems to dwarf those in the former as an industry has been built around prepping under-experienced people for these certifications in the hopes of advancing their career.

This is all to say that the sentiment expressed in this article is unfortunately largely correct. I've since let all of mine lapse because the employers I now work for value demonstrable technical acumen over something that somebody likely bulldozed their way through four years ago and requires $200 annually to renew.

Ymmv: I work in enterprise environment and certifications are often an entry criterion. There's a world outside HN.

i know a major health insurer incentivizes and almost forces IT staff to get at least the entry AWS cert. but now they'll prob tell them to list it on their linkedins so they don't get hired away.

Not even close to being true. I’m forced at times to get certifications for enterprise-y tools and software.

Many networking engineers need a Cisco cert to get started. Same with a lot of security analysts.

If you’re working in Defense you’ll need to get a cert(s) at some point.

Way back being Microsoft certified was all the rage. I was catering at the time near a Microsoft training office and they were a very good customer. The rooms didn’t have a single seat and people were flying in to take them. This was 15 years ago.

Thanks for the context around the subject - piles of data don't mean anything until run through some kind of system and with a goal IMHO. That's an interesting wrinkle in the evaluation / desensitization of employees as humans. Metrics!

What's so hilarious to me in the big picture is how Google and AI in general realize they only make interesting news when they use finished products or concepts because their systems are evolutionally stupid and still can't make a decent song like a child at a piano.

Confirming what we’ve suspected for decades. Certs are a waste of time and money.

Food for thought: So are degrees.

> Food for thought: So are degrees.

I can see why one would disagree with this if they read "So is an education" so I think it's worth pointing out the difference. Degrees are the thing that are corrupted by Goodhart's law (consider the increase in cost of college tuition; people are so eager to spend more because they're trying to hit a target) and an education can only be faked as well as an individual can bullshit.

Link?

Career progress means that Principal Engineers don't have certs -- does that add any value to determine whether certs have value? I'd say Principal Engineers are typically the ones designing the courses and have no use for displaying their qualifications.

Google and google-level companies are a world of their own and I would apply their findings bit-for-bit.

They have a custom way of doing everything, so certifications for the outside world don’t necessarily apply.

Also, unless such findings are shared in a structured manner (in the form of a research paper or something) then it feels like just a cool story, and I’d be inclined to call bs.

To add my personal anecdotal point of view, getting industry specific certifications (red hat rhcsa, for example) has made me a profoundly better professional, because I had to go over a number of topics and study them thoroughly, way over what the average tutorial covers.

Maybe naturally curious people that want to acquire broad and diverse knowledge aren't that interested in climbing the career ladder in Google? Maybe the career ladder in Google is built in a way that encourages narrow focus and singleminded dedication to a single metric only - the career in Google itself? And maybe that's not how everybody is measuring success in life, both professional and broader?

Those are all reasonable hypotheses... but I also suspect natural curiosity has little correlation with certifications. Those programs tend to be big "learn to the test" curricula of trivia and question banks.

I think you made a critical error of being too generic? What field and what certs? If 90% of certs are junk the your takeaway is wrong because people are posting the wrong certs? What about industry? I didn't even know coders had certs but in infosec for example some certs hold a lot of weight.

I've been on the other side of an interview a few times and I can tell you, I will ask more specific and technical questions if you list a technical cert. I have a few certs but mostly to show my range of skills.

See, interviewers look at one thing I did on a past job or a few things I did because they need to be done and their takeaway is "this guy is more into such and such tech but we need someone who knows something else" but I list the certs to allow them to grill me and find out I actually know that shit.

Now, I have seen lazy-ass managers even on HN that blindly associate having certs with not having skills and to be honestI want nothing to do with such people or orgs that tolerate blatantly lazy managers who can't even delegate technical interviews to their subordinates but instead use gut feelings like this to make decisions. A person spends thousands of dollars on certs and their impression is that person is lame but someone half-starts a bunch of things and is a smooth talker and that's their man.

So, yeah, I guess it helps filter out bad employers just like it helps filter out bad employees. If you refuse to interview someone because they actually have a cert, yeah, that shit needs to be illegal.

Read TFA, they do comparisons based on the certifying agency.

I did TFA:

This is what you mean right?:

Coursera LinkedIn Triplebyte Microsoft Amazon Web Services (AWS) Oracle Udacity Udemy HackerRank Cisco

Are you trying to say you can tell what industry the cert is for based on that alone? Microsoft? Cisco? Haha, Udemy????

I think you can infer industry from some of those, but mea culpa; you were focused on a different angle and it was my reading comprehension that was flawed.

People with long About blurbs on their LinkedIn profiles tend to be weaker as well.

really? i found longer texts useful if they contain interesting details about the work of the person. shorter ones always leave me wondering who that person is.

Really. I think this sums it up: https://devrant.com/rants/1807864/reality-vs-linkedin-profil...

Guys who spend a lot on self-promotion are tryhards. The really strong workers don't need to promote because their work history speaks for itself.

ok, yes, i know that joke. but not everyone is doing that, and sometimes those descriptions are just interesting.

As are people with 500+ connections.

We can all make arbitrary heuristics. And we do.

That is interesting data, but it's not that clear that you can derive actionable advice from it — particularly for employee candidates themselves.

Is it the listing of certifications that is inversely correlated with the career progress of the employee? Or is it because listing a certification is strongly correlated to having one, with this being what is truly inversely correlated with career progress? And, if so, in what direction is the cause-effect?

Maybe the tendency to list certifications is correlated to certain personality or attitude that is less likely to progress. Or maybe it is accumulating certifications that is correlated to this. In that case, by not listing them, the candidate will succeed in hiding this fact from potential employers, which may help him or her.

However, what if it is not progressing in one's career that makes one more likely to list (or accumulate) certifications in an effort to compensate? And in that case, all else being equal, maybe the would-be certification-lister would be even worse off were they not to list it!

For a silly analogy, I'm betting that, among users of dating apps, total time spent on them is inversely proportional to the probability of being in a long-term relationship at some point in a given timeframe. However, that does not necessarily mean that using them for long has any causal effect on reducing one's chances.

Data was also collected at Google, which itself is an outlier.

An average business certainly won't have the same recruiting pipeline as Google, to say the least.

Agrees with my own experience. The more you have to proclaim your own competence, the less of it there likely is. As Shakespeare said, "the lady doth protest too much".

That sounds like a shitty job at de-biasing the input data.

This study buckets candidates into certs vs no-certs and then looks at average performance. It might be interesting to look at performance buckets and then calculate the percentage that list certs or not.

> measured as career progress

But that's circular. Career progress is not necessarily correlated with actual performance. The latter is really hard to separate from politics. It is possible to have a stellar career while being a mediocre performer if you know how to work the system and have a bit of good luck. Likewise, it's possible to be an outstanding performer and have a mediocre or even non-existent career if you don't happen to be in the right place at the right time, or if you rub the wrong person the wrong way.

To me, this alone could also be a collider/Berkson's paradox: Conditioning on making the cut, there is a negative correlation because you make the cut by either being extremely skillful or by having great certifications – one compensates the other. That in turn would mean that certifications are actually positively correlated with getting employed at Google.

You need the full picture, that is people not making the cut to exclude this pattern.

Having taken a number of certifications myself, I can say this is not surprising. Certifications have long been used as a means of keeping the labor pool concentrated, and in the interest of limiting those who should be qualified to receive it (based on knowledge) to those that pay the most, there are often structural issues that make certifications as a whole useless to an employer.

The employer is paying a higher price, for no real benefit.

As an example, I've helped IT staff prepare for certification on email systems and have over 5 years of direct experience specializing in that area. They received their certifications no issues, but they paid the high priced business class which included the test. I paid for just the test when I went to test, and I was failed. It wasn't a matter of not knowing the material, there were structural issues that appeared intentional, and there is no incentive to fix those issues. When I say structural, I'm talking about system's theory properties such as determinism not being present (a property any valid test should have).

In fact measures that normal businesses enact to maintain accountability to avoid lawsuits that every business has, they've removed because they are legally shielded with no real due process. They'll string you along for 3 months before giving you a credit, when you can prove they misrepresented many things, and that they made many errors that directly impacted the outcome, while refusing to make right (i.e. you may be limited by the number of times you can test, and errors on their part, by policy, won't be adjusted).

They have such legal protection that you either pay the toll, or you don't get a temporary qualification to work. Its sad, but this interference in employer-employee relations for profit at your expense is common, and unavoidable due to monopoly/oligopoly and lack of oversight.

If I were a CTO involved in hiring, knowing what I know, I would not hire based on certifications. It only shows that you either have persistence or paid more money upfront, nothing more. The marketing being pushed for certifications towards business is fraudulent in many cases, and is not representative of skills.

Do certifications inversely correlate with careers success, or do certifications inversely correlate with age? Older engineers probably don't have heaps of certs because they can fill a resume with prior experience. Additionally, Coursera/Udemy/edX and the like are relatively new and it's more likely that younger people would take more advantage of them. A junior engineer compensates lack of experience by using education and certs as evidence. The key part here is that older engineers are more likely to be senior, and by extension have "greater career success". That doesn't mean there's a direct correlation between certs and success.

I think there's a whole bunch of control variables to worry about before we could take anything meaningful from that data.

> People often suggest that interviewing.io should create a certification that our users can post on their LinkedIn profile, e.g., something like “Top 10% performer on interviewing.io”

Rest of the discussion aside, “we will give you a certificate of being good at interviews” is one of the most absurd grifts I’ve heard in awhile…

If I ever had to interview someone with one of these certificates they would get the most technical, least buzzword friendly interview in human history.

Given how little most people involved in the interview process--on either side--particularly enjoy said process, it seems odd to highlight how much effort you've put into prepping for it.

And it really is like asking for the difficulty to get ratcheted up a level. If I tell you I'm really, really good at interviewing, you're going to take my word for it and adjust your own behavior accordingly. Even if that silly certificate just created an unconscious bias in the mind of your interviewer, with them expecting just a bit more from you, you're a bit worse off.

Shouldn't that be a big red flag? If you are in the top of interviewees, it's a sign that maybe you interview too much (always interviewing, never hired? Or jumping ship at the first difficulty?) or at the minimum a bias to correct for (this person is good at talking, they will seem much better than they actually are).

In any case I really don't understand why you'd want that in your profile.

When you plot graphs that present a 4% point difference as a visual 100% or 200% difference because your axis starts at 50 and ends at 58, I automatically don’t read what you have to say.

ha - i saw this and kind of mentally skipped over it because i was too invested in trying to figure out why this post felt so icky. but, good point. def misleading, which is, as you said, reason enough to stop reading right there.

i think the icky factor is the title. which, it might be a hate the game/player situation -- prob a prerequisite to get some attn these days -- tell people what they should/not do based on your 100% factual, non-misleading, incontovertible 'proof'.

Where I work (we sell software) there's nobody in engineering with certifications. IT has all the certifications.

I've been working since the mid-1990s and this has almost always held true. No one building the stuff has a certification in it. The people who buy it and install it tend to have the certifications.

Over the years I've seen a fair # of hilarious exchanges where someone with a Certification in X tries to tell someone else they're wrong and then gets embarrassed when they are told the person they're arguing with is the inventor or holds the patent on X.

So many of these certifications also seem to train someone to buy products from a company. Our IT guys are always super sure we need products they got certified in, to the point it seems fishy.

My gut feeling is this article is right on the money. The smartest people with formal education are more than capable of teaching themselves this stuff. If you've got a quality degree you shouldn't need a certificate in X to implement X in your product. The certificate won't cover that anyway since no one has ever integrated it into the new product you're building anyway.

I took a C# certification on linked in yesterday. It took 10 minutes and was incredibly simple. And the questions were straight to the point C# language questions, pretty easy if you actually work with the language day to day. I liked that it didn't have any questions you'd only know if you happen to be working with the esoteric JSON library du jour, it was pure C# language spec stuff.

I'll let you know when I'm standing in line at the soup kitchen, but so far finding work hasn't been a problem.

That sounds like the "skill assessment" rather than a certification. I think the former is just a small LinkedIn quiz, and the latter comes from some sort of integration with the cert issuer.

And those "skill assessment" quizzes are meaningless with all questions/answers being a web search away (and not in looking through docs kinda of way but the actual questions/answers).

Berkson's paradox[1] is a useful lens to analyze inverse correlation. The pool of people doing interview prep are either smart or determined (to accumulate certifications) or both, but dumb and lazy people don't enter the pool, and an inverse correlation applies. Similarly, if an interview bar evaluates candidates though a combination of soft and hard skills and rejects those who lack both, even if soft and hard skills are independently distributed (or positively correlated with insufficient strength), soft and hard skills will appear to be inversely correlated.

Certs could be neutrally or even positively correlated with interview performance, but by pre-filtering the population, the opposite phenomenon arises.

[1] - https://en.wikipedia.org/wiki/Berkson%27s_paradox

What's important to understand is that certifications can be used in different ways by different industries.

Some use them as a minimum requirement for performing certain jobs. In such a case, listing certifications is valuable not because it says you are any good at your job, but because it shows simply that you meet the minimum requirements to be legally employed in your profession.

The other way they are used is as a marker of expertise. This is almost always misguided. Advertising one's certification status where none is required conveys a sense of insecurity, need for social approval, and overly strong attachment to formal procedures.

That does not mean you cannot learn something from a certification course - by all means! But true experts are always judged by their results, not by their certs.

I started my career not as a software developer but as a "devops" engineer and don't have a degree.

I found that studying for certs was incredibly helpful for me in the start. I learned how to be a Linux power user with the LPIC-1 and 2, I learned networking with the CCNA, did certifications for configuration management tools for stuff like Puppet, got my CKA, and my AWS SA.

On top of being a self taught programmer (I don't do anything impressive, my code is just throwing JSON from one side of a datacenter to the other) I feel that I am pretty good at my job. I'm comfortable working on a private cloud which has services scaled to millions of concurrent users.

I'll never get a FAANG job, but I still make good money and regularly get positive performance reviews and peer feedback at the jobs I do have.

So if you feel like getting a cert will plug a knowledge gap, please go ahead and get that cert. But don't get certs just to collect trophies to put on LinkedIn.

Also, if anyone can recommend a good alternative to the now defunct Linux Academy, I would totally purchase a subscription. A Cloud Guru ran that amazing resource into the ground and now just caters to cert chasers collecting trophies instead of actually going above and beyond to teach the fundamentals of a technology.

When charts don’t start at 0 I have a sense the author is trying to manipulate the results to fit their narrative.

53 vs 57 is presented as if the difference was over 50% and including shitty Linkedin, Udemy etc. certifications makes me think this article started from the conclusion and tried to find the data to match.

I've worked in finance, healthcare and defense industry, all those industries do not mark you negative for having IT related certifications. Continue education is a big thing in those industries and generally, degrees and certifications usually get reimbursed.

For me, the key is to match certifications with their work. If I see OCAJP, but zero work experience in that space, or you work on-prem system only, but has AWS Professional, then it's obviously useless.

pretty sure they're talking about listing certifications on your linkedin, not just having certifications.

i.e. it doesn't matter to this blog post that you either have or do not have a certification -- it only matters whether you have listed it on your linkedin.

That seems artificial, no? Do I list them on my resume? Wouldn't that have the same "result"?

i'm only saying the report only talked about what was listed on your linkedin, not whether or not you actually had them. not that we should care about being fair to what is clearly a dishonest report.

my linkedin prob lists my expired aws cert. i presume the report only cares that it is listed, not that it is expired, but no idea.

i have _not_ listed my no-longer-valid java cert. the report would presumably ignore that because it doesn't know about it.

other folks presumably list credentials they don't have, or are expired, or were obtained fraudulently, etc.

i'm not hating on their method - just don't care for misleading illustrations - too fox newsy.

and i just realized the headline that i assumed was the title of the post is not actually the title of the page, which means it may not be the intended title of the post. maybe that's standard, or seo chicanery, or clickbaiting, etc.

> Top 10% performer on interviewing.io

That is not a legitimate certification of any kind. If you stick that into a resume, that's a good way to get to the reject pile.

All that is communicating is that the candidate spends a lot of time on some website for people obsessed with interviewing, which is a huge red flag.

A certification is like "completed XYZ training offered by ABC, in 2020", if it's something that has industry recognition.

The author doesn't elaborate a good case on why they would do harm (for readers that didn't go to top universities or have top-only institutional certifications).

For opportunities and recruiters excluding who can't signal top institutions, doesn't matter if you made it to the first screen call because you didn't have them anyway. Forget about them. There is a sea of other opportunities. So no harm.

The bar charts in this article are bad. I guess because if they had normal axis you'd realize the differences are insignificant. Also if I'm reading this correctly, the more certs I list - the less I'll hear from recruiters? If so, excellent. I'll do that in addition to adding a weird unicode character to my name.

you can pry my Andrew Ng Coursera ML cert from my cold dead hands

In past Coursera courses used to have some hard programming-based courses. Now all seem to be quiz-based.

I don't buy the approach. What does it even mean 'Cisco'? CCENT? CCNA? DEVNET? CCIE? Cursera - it varies greatly in quality. Amazon again, Solutions Architect is a memorization exercise, on other side SysOps is more hands on. Yet graphs lumps all those together.

> The engineer who looks good on paper. This person will simply not list the certification on their profile – they have no reason to!

I don't think I buy this logic. I finished an online course with a good test score, and it prompted me to post the resulting certification on LinkedIn. It took 15 seconds at the moment I was happiest about the accomplishment. If we assume I'm an engineer who looks good on paper, am I going to deliberately eschew that prompt?

Is the argument that as an engineer who looks good on paper, I'm in such heavy demand that I never have a few minutes to kill on LinkedIn, in between declining and ignoring the recruiter come-ons? That logic just doesn't make sense to me.

You might have a case, but your histograms are so annoyingly misleading that it's a big red flag for me in general re: your whole site. Would not do business with someone who chooses to make their case that way if I had any choice.

I went to a cyber security college program, and the new program head was turning it into a bunch of micro credentials packaged together with a diploma.

I told him I could self study and get all the credentials on my own faster and cheaper, and that removing labs from the program and changing Cisco certs for CompTIA was not a good idea from a students perspective, and I quit the program.

Dumbing things down to increase graduations rates seemed to have been the goal, and as a mature student that wanted to learn and not just get my foot in the door as a T1 Help Desk role. And we wonder why it's hard to find qualified employees in tech.

As a consultant, I can always tell which engineering consultants are going to be worthless.

The engineers with tons of certs absolutely suck.

Recently worked with one who had like 4 professional grade AWS certs and they couldn’t do anything in AWS

I personally developed an aversion for candidates with multiple certifications on their resume.

They feel to me more like a proof of lack of skills rather than a proof of skills. People with skills tend to not need these, as their track record show for themselves.

I still take them for interviews because some of them got them by accident, as part of their jobs, but I agree they're overall a pretty bad sign.

If you've spare time and want to show your skill, work on personal projects or open source contributions, your time would be much better invested

Certifications are great if they are meaningful. Most aren’t.

A meaningful certification requires a skills based, hands-on assessment of if a person can do the the thing the certification says they can. It should be proctored and require effort to obtain. A meaningful certification is not an evaluation of knowledge but rather an assessment of skill.

RedHat has great certs that are respected in the industry because it tests if you can do the things it asks. You would hire someone based on this.

Many companies let you take a multiple choice test to “certify” you. Those are worthless and are generally just part of the marketing funnel.

So the theory according to them is that if you don’t have a good company or school on your cv then you don’t have pedigree, so though luck, don’t pretend you’re good because you study and pass exams. This from the lady who was calling developers ‘peons’. If you put a certification ‘top 1% of interviewing.io’ then yes, it’s worthless. But if you have a industry known and reputable certification, then you stand out a bit, depending how fresh it is. You still need to pass the interview.

Wondering if HN can give me a hand. Been trying to segue from dev to devops (or platform eng) and certs seem like a good way to indicate interest and some level of competency in lieu of job experience. Had devops interviews before any nobody wants to take a chance on me or feel like it'll cause problems since I am mid career as a dev. What are my avenues if not certs to get my foot in the door? Targeting AWS. Have a dev blog and github with projects.

The premise doesn't follow conclusions here. There is a big difference between a certification that says that "I know JavaScript" versus "I am in the top 10% of JavaScript developers (by whatever metric).

Coursera/Udemy/etc certs are obviously useless because they don't actually signal anything.

A "Top 10% at interviewing" cert is a bizarre idea but the two don't compare imo.

The one thing where I disagree is Cisco certification. CCIE, at least back in the day, was the gold standard for network engineer. It was pretty much required if you wanted anything to do with networking. I’m not sure now if that’s relevant but I think it can’t hurt.

Programming certifications definitely look terrible. I would rather interview someone with a side project than someone with a cert.

"The one standout is Triplebyte. Their graduates are 6 percentage points more likely to pass interviews —a serious boost, albeit not enough to dispel the negative signal that all the others carry."

I believe that interviewing.io and Triplebyte are associated entities. So grain of salt.

Aren't they competitors?

One thing I've been thinking about is getting a GCP certification to increase GCP recruiter reachout down the road, as I've really enjoyed working with GCP and would like to stick with it in future work. Guess I'll just do more keyword work in the LinkedIn. :)

Exactly. I'm an independent consultant and I just got my AWS Pro cert for marketing reasons.

And how is it going, did it make any difference?

I tailor my CV to the job I want. The golden rule of two A4 pages with the relevant badges and certs which apply to the job you want, not a laundry list of every little badge you earned throughout your career. There is sometimes merit in simplicity.

From the article:

> today’s depressed labor market

I thought the US unemployment rate for tech jobs was below 2% (unemployment in general in the US is below 4%).

> If your profile doesn’t have a reputable school or a top company on it

Which is like 99% of the entire job pool? What's a 'reputable school'? Ivy league? My university was abroad. It's reputable there. It's unknown in the US. Do we even care about schools in IT after a couple of years in the job market?

Likewise, what's a "top company"? FAANG? Fortune 500?

But yes, 'interviewing.io' certification would be BS indeed. All it would signal is that the person is good at interviews. It would be even worse than leetcode.

Interesting take. Post title should be more precise, the article focuses on engineers’ roles.

That graph seems like something straight out of the Lying With Statistics book.

I laugh at certifications, people that accumulate certifications, am skeptical of adjacent industries that actually seem to respect certifications (IT, Cybersecurity) and still sort of laugh at them because actual programmers and hackers defeat the "best practices" all the time while having no certifications. I'm also skeptical of bootcamps the same way.

I don't have a solution to gatekeeping, I'm not saying "you need a university degree", because you don't.

My feelings do match the model shown in the article.

Reply on news.ycombinator.com