Hacker News Reader

Comment by orev | Hacker News Reader

There is a huge problem in IT. It's not certifications. It's the totally illogical bias against certifications. There's no reason someone can't have both skills and certifications, but everyone treats them as mutually exclusive. Certs help with administrative things like HR requirements, contractual obligations, audits, etc... No, those things do not make one secure, but running a business is not only about being secure. The problem most IT people have is thinking that certs address technical issues, when in fact they address business issues.

View on HN

> There's no reason someone can't have both skills and certifications

Of course you're right that it's not impossible. But here's why it happens anyway and why the heuristic of them being roughly mutually exclusive is not insane:

1. There's a certification that's nearly meaningless because it's so easy to obtain without also having the relevant expertise that the certificate is supposed to represent.

2. People who are actually good at the thing will notice the certificate doesn't measure the skill correctly, and will also note that there are people in the world with this certificate who don't know the skill.

3. Those experts will not use the certificate when they hire people, and will not get the certificate since it doesn't work and no one who is an expert is using it to hire anyway.

4. Meanwhile, there are basically only two groups that care about the certification:

> a. People who are clueless: a clueless hiring manager who doesn't understand the domain they are hiring for so they are looking for cheap proxies for skill and experience. Clueless wannabe professionals who don't understand the domain or the industry enough to have been in the expert group above but who are still looking for jobs in the field. Clueless clients who are impressed by the certification because they don't know any better.

> b. The people who are taking advantage of the clueless clients: Professionals and hiring managers who know very well that the certification is worthless, but who use it for sales and marketing anyway because it mollifies clueless clients.

If that trend holds, then you have a signal (the certification) the repels experts, while attracting the clueless and those who would exploit them.

So when you find someone in the world with that certification, you should expect on average for them to be clueless or preying on the clueless. That's why treating it as mutually exclusive isn't insane heuristically, even though it's not impossible that someone who is good also has a certification.

Additionally technical certificates supposedly measure competency in a particular technology. That technology is obsolete roughly at the time you're sitting to write the exam. So you've spent time learning material (assuming you didn't already know it) to cover a technology that will be in the process of being replaced as you sit down to write the exam. You may as well have spent the time using the skills productively rather than using it to obtain a piece of paper that says your skills are obsolete...

Plus there's the financial outlay of it; costs to obtain the course material and sit the exam.

These points (including yours) basically amount to my bias against obtaining certificates.

I'm not against the material itself, if it's useful. But I'm not in the habit of paying for or spending time on education for anything other than the value of my interest in the material. A piece of obsolete paper does nothing for me at this point in my career.

YMMV

This is a great comment to illustrate grandparent's point that irrational hate of certifications permeates the industry. Many certifications under discussion here are meant to assist at the entry-level. That's what the certificate is "supposed to represent". That's what skill the certificates "measure correctly." That's the 3rd group of people who care: people operating at or near the entry level.

It is no surprise that experts hiring experts in any field couldn't care less about certifications (aside from ones required by regulation). At the expert level, you hire real, verifiable experience.

You've ignored the point of the post you're replying to. You're looking at the credential as a employee signalling tool, not a tool for other parties to satisfy a business need.

Your HR department needs avenues to sift through referrals and comparison points. If an individual has the certificate and compares equally with a non-certificate candidate, the first individual has signaled, through the certificate, that he is interested in the field, as well as willing to invest time and resources into advancing in that field. This is the flipside of the employee signalling.

If an applicant applies without the certificate to a post which requires it, he will recognize his cover letter/interview should make a point of demonstrating competence in the area of the certificate. This is an instance of employer signalling.

If an applicant fakes having acquired the certificate and there is an easy way to determine whether or not he is faking, then HR now has an easy sieve for removing employees willing to misrepresent their skills to obtain the job. This is a step in the HR QA process.

Some employers work outside the 'work for hire' states. If they contain a certificate requirement in a job posting with enumerated skills, they can point to ineptitude in the certificate skillset as reasons for dismissal depending on the jurisdiction they're in. This is a tool in legal's toolbox.

Perhaps the certificate system is the result of the collaboration of a number of employers attempting to pool credential and training requirements for an industry, then create requirements for credentialing into legislation to control the industry pipeline of workers. This is a method of signalling to legislators or restricting competitors by indirect means.

And so on.

Additionally, your fact pattern has a small problem: expert acting as hiring managers who have knowledge of the problems with the certificate are free to adjust their screening procedure to obtain more fully vetted candidates. The only time avoiding the certificate entirely is when the signal it provides is negative.

This doesn't mean that certs are useful, merely that they might be useful to stakeholders you aren't considering.

I did ignore that mostly, you're right. My basic claim is that on average this is actually true:

> The only time avoiding the certificate entirely is when the signal it provides is negative.

And further, I think all your examples are perfectly valid, real-world examples that I don't dispute exist and that lots of people find important. I also think perfectly good and reasonable people operate in the reality of their industries and play ball with these things when necessary. AND I think all the use cases you mentioned are bad for the system overall. As in, they are real, and in a practical sense we can't just ignore them, but ideally we wouldn't have them.

HR using negative signals for filtering is bad. Job requirements tailored to the actual job are good. Broad and mostly arbitrary requirements from a third party are bad. Applicants to a well specified job also know to address weaknesses in their cover letter. Specific and well-specified requirements are also useful in legal disputes. Using arbitrary requirements to provide legal for firing people is bad. Employers colluding to control the training pipeline using arbitrary requirements encoded in law is bad.

I agree with you that they useful in the real world, but I'm arguing that that usefulness is evidence that the system is worse than it could be.

> Your HR department needs avenues to sift through referrals and comparison points. If an individual has the certificate and compares equally with a non-certificate candidate, the first individual has signaled, through the certificate, that he is interested in the field, as well as willing to invest time and resources into advancing in that field. This is the flipside of the employee signalling.

The real root of the issue here is that HR can't do their job. An unintended side effect of all this signalling is that future employees will be glad to get filtered out because they don't want to work somewhere that hired on certification instead of competence.

> The only time avoiding the certificate entirely is when the signal it provides is negative.

It is possible for the same action send a positive signal when done by some people and a negative signal when done by others. Specifically there can be 'contersignaling' [0], which is basically signaling that you don't need to signal.

[0]: https://kelley.iu.edu/riharbau/cs-randfinal.pdf

> Certs help with administrative things like HR requirements, contractual obligations, audits, etc.

You're pointing out things that are generally considered to be failures in our sphere anyway. I don't think that's unrelated.

HR brings us terrible candidates? Certs (currently) don't help that, and when HR over emphasizes them, we can blame the certs. (In theory, certs COULD help, but the people that get the cert instead of the experience are the problem ones, and those are the people HR brings us)

Contracts are an attempt to preset agreements between two parties, and in the tech world usually one of those parties are at a disadvantage in the tech space (otherwise we'd not be making the contract). Certs are used as a means of asserting competence in the employees, but the other party has every incentive to focus on the cert instead of the competence. So when it goes bad (as it often does), we can blame the certs as not achieving the goal.

Audits are likewise, trying to filter for competence (in action or in people, depending). But again, if the incentive is for the cert over the competence, it is the competence that suffers. We blame the cert (or more correctly, the emphasis on the cert).

IT definitely has a bias against certs. And it's not really against the certs themselves...it's that emphasis of the certs is harmful, not helpful. So we argue against that emphasis, which sounds a lot like hating the certs.

> The problem most IT people have is thinking that certs address technical issues, when in fact they address business issues.

But those business issues are "how do we ensure the technical issues are addressed?" And it turns out Certs in practice do a terrible job at that.

It sounds like when the poster said "address business issues" they meant "enhance regulatory/bureaucratic issues."

The problem is people outside the field of whichever cert pile into them as a means to break in, so the median cert holder winds up being inexperienced and overall unqualified. The cert itself then becomes associated with that. You're not wrong that its silly to be biased against, in the larger context of someones qualifications, but only certs that are sufficiently exclusive will get any respect and that probably isn't going to change.

Yeah, when I started years back I had the Security+ as it was a requirement by my university at the time. During an interview I had the technical manager ask me about it and openly wonder why I would waste my time with such a useless certification. I didn't get that job, and I still wonder if it's because he thought less of me because of a certification I was required to obtain in order to graduate.

I leave my certifications off my resume now, unless the job posting asks for certifications which is frustrating trying to decide if adding this cert to my resume will help or hurt me in the interview.

There's both some truth and some falsehood to the 'certifications don't prove anything' argument:

Answer a multiple choice test for an MCSE or whatever? Doesn't prove much.

Receive a server that's been wrecked and won't boot, turn it into a load balancing HTTPS server, SMTP server, a bunch of required cron jobs and a boat load more requirements for RHCE? Proves you can do those things.

Disclaimer: used to work at Red Hat. Still love their stuff. Cisco has task-based tests too.

Yeah it's just a shame that multiple choice tests are the standard rather than practical examinations. All the multiple choice tests look for is if you have memorized their documentation and can figure out the test format rather than if you can actually apply it, which are different skill sets.

I'd agree that the problem isn't so much that certification has to be bad, but that bad certifications (e.g. those that examine though purely multi-choice) aren't appropriate/good for the industry...

For me the answer is better certifications.

I just recently had someone ask me to take an assessment test for a senior developer position. There's always some silly hoop to jump through, so I thought "why not".

Well I got booted out of the test because I hit Ctrl-C to copy something for the first warning, and hit Ctrl-L (muscle memory) for the final revocation of the test.

I just thought to myself ... did I just fail an assessment test because I hit Ctrl-L?

The test gets reset and allows me to start back up, only it took 20 minutes off the allotted time for some reason. I didn't even get to the last question on the test.

The results showed me scoring in the 96 percentile. So I basically threw 3 questions away on this test and still scored that high. And I skipped one question because it was asking about building/creating msi files on an C#/ASP.Net/MVC assessment. I have no idea why it was there, but I don't regularly build msi executables (although I regularly automate them).

And the worst part is that a lot of the questions were inane things like "given this inheritance hierachy, sally adds the new keyword in front of one of the child methods, and then this other code uses this inheritance hierarchy. what is the output?".

At no point do I feel like anything on that assessment came even close to assessing my ability as a senior technical person. These were things a college student could have answered just as accurately.

I know it's not quite the same thing as a certification, but I seriously dislike assessment tests. Unfortunately it seems like every company has their games you have to play in order to actually get TO the technical folks.

I think I've done this test or a similar one. The worst was going for an MVC role and it was giving me a bunch of questions on webforms lifecycles and "what does the X.Y" method do.

I've had several of these in my job search recently as well, with a background in education I'm astonished that anyone thinks they show anything.

What you are saying points out problems with contracts, audits, HR. Audits are supposed to meaningful. They aren't supposed to be a waste of time that exists to check off a box. HR is supposed to add value, not subtract. Counterparties should ask for things in contracts that actually benefit them.

From a certain point of view you are right -- an IT manager should cooperate with other facets of the business in order to help the organization succeed, not obstruct because he thinks their requirements are dumb. But from a larger point of view, either organizational or super-organizational, dumb requirements shouldn't exist. To the extent that they do CEOs, industry groups, and/or governments should be modifying them to keep them relevant.

   There's no reason someone can't have both skills and certifications

Ok, sure...But we're not saying they are mutually exclusive. We're talking about bayesian inference here...

Especially if the person comes from the government. CEH is garbage, but it's also a cert the US gov selected.

I had to get the CEH when I was hired on by a security consulting firm back in Northern Virginia.

I now do not include this cert on my CV for (perhaps irrational) fear that someone in some HR department may think "Oh, no! This guys is a hacker!" Sadly, the word has a negative connotation, because the word "cracker" or "bad actor" never bubbled up past the IT security world.

I also do not include my military service dates, as they reliably peg my age. Most men join at 17 or 18, so they would immediately know my age.

I'm debating whether to include any certifications at all, just include my degree.

Technology changes so fast that most of this tests are obsolete by the time you take them. The best technologists rely on their knowledge, so they spend their time and efforts staying current. On the other hand, you have weak applicants who know a certification is their only way to a job so they put their efforts into it.

Nope, it is in fact the certifications. Strongly agree with the tptacek quote here.

So how would you approach scaling the IT Security industry without some form of industry certification process?

I'm definitely not trying to defend the CISSP/CEH style certs here but I don't see how you reliably expand the industry at scale without some form of certification process.

A company hiring it's first security person or a company trying to hire a lot of security people, need some form of base benchmark to work from, just like with most other professions (e.g. Law, accountancy, architecture etc)

What does "certification" have to do with scaling the industry? Training and nurturing talent is a hard problem, but expensive tests don't do anything to mitigate that problem.

Certifications are used by many industries to provide a demonstration of a common baseline level of knowledge and experience, so that each individual person doesn't need to be assessed by each hiring organisation.

For example Certified accountants, Lawyers etc.

Without some common baseline, how do people looking to hire security types who don't have the experience to assess their skills and knowledge avoid getting bad people?

Also the article's argument that "it's experience that counts" really doesn't help get new people into the industry, where do they get the experience in the first place, it's a catch-22

Apprenticeship works. Lots of places will have junior people work below senior ones on projects and gradually gain experience and become senior.

Yep I think apprenticeships can help too, there's no one thing that's going to help bring a load of people on, I think it's got to be multiple paths.

I believe that too. I simply believe --- with ample evidence --- that certifications aren't going to be one of those paths.

so, out of curiousity, that implies to me that you don't rate any IT security certifications?

So would I be right in thinking you don't think that any of the Offsec certs (OSCP/OCSE), CREST certs (CCT etc) or SANS certs are usful?

Also, and I'd be genuinely interested to hear your thoughts here, why do you think that IT/Info Sec will take a different path than other professions (medicine, law, accountancy, engineering etc) which fairly universally have evolved into a certified professional model?

The "certifications" in medicine and law accompany postgraduate degrees and are far, far more recognized than the random certificates you listed, some of which are profit-making enterprises from for-profit companies.

Ah ok, so would it be fair to say that you're not opposed to the concept of certifiation, per se, but that you're not a fan of existing options in the field?

Of course one problem is "how does a certification become recognized", I mean in IT security it's going to have to start somewhere...

In the UK the IISP are perhaps closest to the "traditional profession" certifications, but they're struggling a bit to get traction.

You can't wish professionalism into being. You have to build a profession. We're not there yet with any aspect of information security. The hard work of defining the field and its requirements has not yet been done. No organization currently extant on this planet has any business pretending that they know the answers to these questions, let alone charging money to take tests about them.

Obviously it takes time to build a profession, but you've got to start somewhere, and part of that path is certification.

Unfortunately the industry is growing far faster than perhaps happened for previous emergent professions, so the time needed to slowly grow professional bodies isn't available.

If it's not commercial organisations that start providing those services, the only other options I can see are some form of union, or some government mandated body. Those are options, but both have their challenges.

Both those options have their downsides.

No, you're describing a cart that is pulling its horse. The "certification", in whatever form it takes, must follow the professionalization of the field.

Regardless, none of the certificates you've mentioned --- OSCP, CREST, or SANS --- will define information security. None of them have any meaningful credibility to experts.

Interesting, so what's your view of how professionalization of the industry should get started?

The burden of proof should be on whoever is suggesting that security has anything at all to do with those professions, but I'll throw out a couple of observations anyway.

Those professions have rules, and are backed by either legislation or science. All participants are bound by said rules. For a lawyer, certain things are legal, certain things are not.

Security is a game where the whole objective is to either break the rules (and often the law) or to defend against someone who is.

How are you going to tell me that person A is qualified for the job based on his exam results, and person B is not, when person B got a root shell on your server and stole your data?

It's like the 1989 draft when the Giants tried to make Dion Sanders write an exam to see if he was qualified to play in the NFL.

"“They sat me down and gave me a thick book,” Sanders recalled. “I mean, this thing was thicker than a phone book. I said, ‘What’s this?’ They said, ‘This is our test that we give all the players.’ I said, ‘Excuse me, what pick do you have in the draft?’ They said, I think, 10th [actually 18th]. I said, ‘I’ll be gone before then. I’ll see y’all later. I ain’t got time for this.’ That’s a true story."

It's difficult to believe that anyone who can claim to really know computers thinks they aren't based on a series of interacting rules. That's basically all they are.

Understanding how those rules interact, how to trigger certain interactions others didn't intend, and the best practices to not get bit by those interactions is what security is all about. It's much like law or medicine in that you are looking at unexpected consequences of multiple complex systems interfering with one another and looking for compromises and best practices to keep the most disastrous interactions the least likely to happen.

Computers follows rules the same way a football player follows physics. Those are not the rules GP is talking about.

No. Computers are rules. They are multiple, complex, partially abstract and partially concrete rules systems with messy and often poorly defined interactions among those systems.

The individual NAND, AND, NOR, OR, and/or XOR logic? Rules - tabular even. The base IA? Rules. The microcode? Rules. The VMM? Rules. The TLB? Rules. The assembly? Rules. The OS kernel? Rules. The C library? Rules. The ABI calling convention? Rules. The application language? Rules - syntax and semantics. The libraries under the application - rules. The application itself is a list of rules for how data is processed. If it's Turing complete it's basically equivalent to the lamda calculus.

Every security issue is some misapplication of these rules due to someone not understanding the implications of the interactions of the rules. Every single one. Smashing the stack? It's applying a set a rules in a way the code author didn't anticipate. Overflowing a buffer? The code author didn't anticipate more data being stuffed in than the buffer was made to hold. Rowhammer? There are rules of semiconductor electronics interacting with the programming language, the IA, and the logic layout. SQL injection? Someone's applying the wrong rules to sanitize the input and someone else is giving input that takes advantage of the underlying rules of the programming language and the RDBMS that they were allowed to invoke because proper sanitization wasn't in place.

There is not a single person on this forum who needs your lecture about how, in one sense of the word, computers are based on rules. The obviousness of this fact should indicate that the disconnect is elsewhere.

The rules under discussion are policy, and the question is how to define policy for evaluating people's skill at running roughshod over policy. The fact that there are underlying "hard" rules is literally universal and therefore uninteresting in this context.

I really don't think you're following the point.

Lawyers and doctors deal in interacting complex systems of rules. So do information security people.

If you can make a certification that works for one expert in dealing with interacting complex systems of rules, you absolutely can make a certification for another expert in dealing with interacting complex systems of rules.

The details of what you test are different, but the fact that it's been done for law, medicine, medical specialties, dentistry, mechanical engineering, electrical engineering, civil engineering, and many other fact and rule based fields means it can most likely be done in general for people looking at how different systems of rules intersect.

No, you can't. That isn't how security works. Your offensive adversary follows no rules. They exist to break any rules you can think of. You can make all the rules you want and you can test people on their knowledge of them. Hackers do not care.

On the defense side, they simply do not work. Everybody gets hacked. The best companies with the biggest security budgets employing people at the cutting edge of security research still get hacked. Security experts get hacked. If the best in the industry still haven't solved this problem, you can't even begin to make the framework that you're proposing.

The discipline cannot be described as experts dealing with interacting complex systems of rules.

People still die. People still lose lawsuits. I-85 still partially collapsed. Yet people can be certified as knowing and following best practices in those fields.

Illness doesn't follow rules the doctor dictates. Rainfall and earthquakes aren't set by the engineering boards. How people adjust, prepare, and respond are where the rules humans can set work. Everything else is unchangeable rules.

In computers, as in some parts of law, we have ample opportunity to address underlying rules as well as the rules around how we adjust, prepare, and react.

The challenge that I've been trying to express is that individual excellence is hard to replicate at scale. Sure in sports, that works ok, the numbers are small and the rewards are great, the incentives are there for people to comb through thousands of people to find that one candidate who's truly excellent.

But that's not the reality for most companies, they're not trying to hire the absolute brightest and best, realistcally not everyone can. They're looking for some measurable indications that a candidate has a baseline level of knowledge in a given field.

Now I feel that a good certifiation can be part of that. So a valuable activity for the industry is to try and create better certifications to help companies who aren't in a position to judge for themselves whether someone is great at something or not, that there's a level of knowledge and understanding there.

If current certifications are poor, then it should be possible to articulate why they are poor, and describe what would make them better.

My references to other professions were designed to reference the fact that those professions have had to face similar issues as they've grown and in science, engineering, law, medicine, accountancy, etc etc they've pretty much all decided that some forms of professional certifications are the right way to go.

Not to say that they're perfect, but that they could be better than the alternatives.

How many of those industries does certification actually work in? Does it prevent dodgy lawyers and accountants?

Certification does work in some industries.

When a lawyer is admitted to the bar, that is generally taken as proof that they have some idea of what they are doing. Bar exams are hard.

And the bar also provides a forum for dealing with shady lawyers. If a lawyer treats you badly, you can file a complaint with their bar association and they might get disbarred. This is an area where cyber training and certs is not as good yet, I think: as a forum for resolving disputes.

TV may be leading me astray, but isn't disbarment usually for ethical reasons, not general incompetence?

The problem is one of trust, methinks. Most certs require only that you "know" the material well enough to pass a test that uses your memory. If the tests were empirical, say like the Red Hat tests or the Cisco CCIE, then the trust that someone actually has skills might be more believable.

Having worked in IT security for many years, I can attest to the fact that IT security is more of a subjective set of processes rather than a specifc product or set of products.

There are skills involved as far as tools and knowledge of how to use tools, but these change depending on the use case.

Yep the problem is that some of the well known certs are bad, not that the concept of IT security certifications are bad.

I've been in IT/Infosec for 17 years now. I have certs that I literally maintain as a HR/sales checkbox (e.g. CISSP) and certs that make me think every time I need to refresh them (e.g. CREST CCT). It's possible to have good IT sec. certs.

Reply on news.ycombinator.com