Security Certifications Are Causing More Harm Than Good | Hacker News Reader

Security Certifications Are Causing More Harm Than Good

tacnetsol.com

290 pointsginasilvertree215 comments

The thing with infosec is that no matter if you're a consultant pen tester or an in-house member of a blue team, a high proficiency in technical writing is required. And few certs demonstrate that the person is a good technical writer. It's not enough to know the answers to multiple choice questions. It's not even enough to know how to exploit things. If you don't understand something well and can discuss it in technical detail to a number of different audiences, I don't believe you'll get very far in the industry.

There are a couple of exceptions, of course. OSCP is a good certificate to have. To pass the exam, you are required to not only demonstrate proficiency in several areas (i.e SQL injection, buffer overflows), but you must also write and submit a technical report to a review team. The technical report must address vulnerability overview, impact, risk rating, reproduction steps, and more. Of course the exam isn't perfect, but it's probably the biggest test of real technical understanding and ability I've ever seen.

nice to see OSCP still regarded well. I got it about a decade about when our CTO had an initiative that everyone doing customer interaction (R&D team + professional services support for me at the time) should have a cert.

I vehemently disagreed, pointing out that those of us with meaningful degrees from well regarded programs shouldn't have to get one, but in the end I complied to keep the peace.

Have to say I enjoyed it at the time and even learned a thing or two. I wasn't exactly new to offensive work, but was nice to get things sharpened up and familiar with a few different approaches. I remember being a bit worked up about the final challenge at the end of it that I took the day off work, but iirc I was able to finish in just a couple hours. It wasn't the most cutting edge content at the time, but it was well organized and ensured that the person being certified could demonstrate some level of practical application and that is far better than most certifications of any sort.

This. So much this. Writing and general social skills are the number 1 thing lacking with people I interact with in the industry. The I know more than you attitude is great amongst peers, but with clients, you don't have to prove you're smarter, instead your job is to make them smarter.

Break it down to a 4th grade level, if you can't, you likely don't understand it yourself.

I would agree that in general certs have too much weight, but the reality of it is, as it stands, they're the closest bridge / standardization that the market has built for non-Security ilk.

Also, Security is not IT. That's another thing that needs to change.

As for OSCP, I think it's great and out of all the certs i've taken for various reasons, it's the only one I felt challenged with, in a good way.

Also, Security is not IT. That's another thing that needs to change.

Do you mean that

    Security is not currently IT and it should be

or do you mean

    Security is considered to be IT and it shouldn't be

? The amount of stress that you mean to put on the word "not" doesn't come through very well in this medium.

I feel like he means it's generalized into the spectrum that falls into IT. Security should be more of specialization, less toolbox?

Security should be IT. Otherwise, you get a bunch of bullshit cya reports and policies that aren't achievable by the technology deliver people.

Security should have an advisory role to the corporate governance folks who maintain policy... usually the lawyers.

Knowing basic English is a prerequisite to any tech job. There's nothing else you need to know to explain a bug. And there's TOEFL/IELTS if you're looking for English language certificate.

In theory, maybe, but there are many people who are fluent in basic English but who can't usefully communicate technical matters like bugs (except, perhaps, to someone who already looked at that specific problem with them).

Edit: And, in contrast, I've met people who I had a hard time communicating with in spoken English but, had no problems with once we did our communications in writing or with a written point of reference.

It's a prerequisite until the powers that be realize they can save money by dropping it.

Communication in tech, and especially infosec is a dramatically underrated skill. Infosec consultant for 10 years. I write. A lot. Being able to jump from explaining how we reversed something to devs to an executive who just wants a certain view of how that impacts a release is hard and has taken me a long time. You essentially have to understand the various consumers of your writing at a pretty deep level for it to get read and have impact.

A well written report that speaks at the right level to its audiences will generate more proactive security activity than a terse, passive voice bomb of dense technical information.

It's unlikely that typical OSCP-holder could write a modern buffer overflow exploit, or even judge exploitability of a memory corruption flaw given the source code and a traceback.

Equally importantly: memory corruption exploit development and SQL injection are different skills, and most people who do SQL injection don't need proficiency in "buffer overflows". Why is superficial coverage of "buffer overflows" part of the rubric for that certificate? I don't know, and I don't know that anyone else does either.

Is there a single coherent security certificate anywhere in the industry? I'm interested in examples.

If you want that wouldn't you want OSCE instead of OSCP?

Totally agree with this. I think security / infosec is just seen as a small niche, so people lump the cryptographers, auditors, SOC analyst, malware analyst, appsec, incident response type people all in one group. It's hard to find someone who can cover all that ground proficiently. Of course it's also been my findings that people who end up in infosec tends to be generalists, but I wonder if that is shifting now that you see more cyber security school initiatives.

Total tangent but, I am absolutely grossed out by "cyber" winning out in the name game. Who let the DoD drive that? Damnit!

This is why when I was in an infosec bootcamp I begged you to talk to my class and give a dose of reality.

The leet kids, a minority few and the rest naïve, I would bribe while they whittled away at online CTFs and MicroCorruption and irritate them with mediocre questions until they tuned me out. I did not care for tools; approach and mindset are order of magnitudes harder to explain.

I thought you could be a wakeup call had you told them all the certs pro se is a waste in a program that pushed that nonsense. I talk trash of my certs and skills the whole time and they did not get why.

I know you're busy, but I've read your blog and you're preaching to the choir. Starfighter folded, but I would pay for you to test me as a customer and find a mentor to answer my stupid questions that I would pay handsomely for the privilege. I feel I'm not the only one, if you get tired of NCC, that would be amazing!

I'm not at NCC!

I totally agree that most web pentesters don't generally need to know how buffer overflow and binary exploitation techniques work but I think an understanding of how low level systems function and how they can be exploited is useful across all security sub-fields.

I don't think Offensive Security is trying to pump out exploitation experts from their entry level cert program. Maybe the higher levels OSCE and OSEE. The intro cert emphasizes breadth over depth. It felt a lot like a cert built around the Exploitation Hackers Handbook.

I think you're thinking of the certs in the wrong light. They are meant to validate baseline knowledge and proficiency, not mastery. If you want to validate mastery you need to look at the persons personal record and work product.

What you think about GIAC Platinum, GSE, or whatever it was called? And with a mix of software, networking, and incident handling before the interviews and pentests?

It's the only one I ever thought might be valuable since it combines specific knowledge with some pentest against real systems. Curious what you think as you're more qualified to assess such a thing given all your security evaluations and hiring those people.

> The thing with infosec is that no matter if you're a consultant pen tester or an in-house member of a blue team, a high proficiency in technical writing is required.

As much as I admire people who show the courtesy to the trash bin that will eat their report, before any human reading it, of not feeding it "bad" reports: The OPs point of "sense of false security" unfortunately already sets in once someone is hired to "take care of security". This anecdote from about 2000 illustrates that:

A friend was hired to do black box penetration testing on the DMZ and internal network of one of the largest travel agencies in Europe. He, of course, found a lot of problems. He wrote a nice report. Like really nice. Point for point "this is your problem, this is what you have to do right now and this is a user-friendly policy you could implement to prevent such issues in the future." It was very pleasant to read, he knows how to handle language. And somewhere on page hundred-something "the first one to claim it, will get a bottle of champagne!".

The bottle was claimed a few months later by someone 3 levels above his position. By some high-level manager who did not know anything about IT. But he was the only one to read it (and he only claimed the bottle to figure out whether anyone below him had actually read the report). Two years later company politics allowed those 2 illiterates between my friend and the one manager who could read to be removed from the company. My friend was hired as their chief of security. His first task: work through his own report to fix the 90% that had not been fixed since he wrote it 2.5 years before (the report listed passwords to core routers in plain text as examples for "very stupid passwords" - they all still worked).

One of the most important aspects of a long report is the prioritization of the contents.

I used to pentest for a living. Still do some red team exercises every now and then, but far less now that I'm mainly blueteam focused.

I personally organized my report into three sections, which seemed to work well. Clients seemed to enjoy the formatting:

1. Executive - Summarize everything in one page at a high level. You could skim it fast if you chose to. Highlight potential negative business impact of each finding.

2. Management - A little more detailed. 2-3 pages max. Most severe findings at the top and recommended action for remediation.

3. Narrative - This is the bulk 80-90% of the report detailing your step by step process including screenshots so that if someone wanted to duplicate your findings they could.

The last (and only) pen test report I saw had some "escalation of such and such via this vague vector, please do x and y to close off this avenue" - strongly implying that they felt the actual trick being used was their IP to be protected?

This was for using citrix boxes to provide saas to a client

About a decade ago (give or take a year) our small web company had a PCI test against our network and a 500-page PDF ended up in my email. The "report" was obviously the vomit of a program detailing every issue for every site [1]. You would think that after the 50th time our DNS resolvers were "open to the public" (as it were) that the stupid program vomiting the report would realize it's reported the same DNS resolvers each time.

[1] It didn't help my perception of the report when it screamed that "ICMP echo was enabled and nefarious scalawags might be able to do unspeakable acts against our computers, best cut the network cable" type of advice [2] (yes, I know port 443 is open! WE'RE A WEB HOSTING COMPANY SERVING UP COMMERCIAL WEBSITES! ARE YOU PCI AUDITORS STUPID?)

[2] Okay, the ICMP echo thing was reported, but did not need to be disabled to pass the audit. If so, why even bring it up?

Maybe someone read it but was a teetotaler?

At one company I'd do a similar thing - about two-thirds of the way down a list of dot-points, I'd mention something about free chocolate or a beer. No-one ever mentioned it, not even to reference it in the slightest or tell me I was wasting my time.

There is a huge problem in IT. It's not certifications. It's the totally illogical bias against certifications. There's no reason someone can't have both skills and certifications, but everyone treats them as mutually exclusive. Certs help with administrative things like HR requirements, contractual obligations, audits, etc... No, those things do not make one secure, but running a business is not only about being secure. The problem most IT people have is thinking that certs address technical issues, when in fact they address business issues.

Especially if the person comes from the government. CEH is garbage, but it's also a cert the US gov selected.

I had to get the CEH when I was hired on by a security consulting firm back in Northern Virginia.

I now do not include this cert on my CV for (perhaps irrational) fear that someone in some HR department may think "Oh, no! This guys is a hacker!" Sadly, the word has a negative connotation, because the word "cracker" or "bad actor" never bubbled up past the IT security world.

I also do not include my military service dates, as they reliably peg my age. Most men join at 17 or 18, so they would immediately know my age.

I'm debating whether to include any certifications at all, just include my degree.

Yeah, when I started years back I had the Security+ as it was a requirement by my university at the time. During an interview I had the technical manager ask me about it and openly wonder why I would waste my time with such a useless certification. I didn't get that job, and I still wonder if it's because he thought less of me because of a certification I was required to obtain in order to graduate.

I leave my certifications off my resume now, unless the job posting asks for certifications which is frustrating trying to decide if adding this cert to my resume will help or hurt me in the interview.

The problem is people outside the field of whichever cert pile into them as a means to break in, so the median cert holder winds up being inexperienced and overall unqualified. The cert itself then becomes associated with that. You're not wrong that its silly to be biased against, in the larger context of someones qualifications, but only certs that are sufficiently exclusive will get any respect and that probably isn't going to change.

What you are saying points out problems with contracts, audits, HR. Audits are supposed to meaningful. They aren't supposed to be a waste of time that exists to check off a box. HR is supposed to add value, not subtract. Counterparties should ask for things in contracts that actually benefit them.

From a certain point of view you are right -- an IT manager should cooperate with other facets of the business in order to help the organization succeed, not obstruct because he thinks their requirements are dumb. But from a larger point of view, either organizational or super-organizational, dumb requirements shouldn't exist. To the extent that they do CEOs, industry groups, and/or governments should be modifying them to keep them relevant.

Technology changes so fast that most of this tests are obsolete by the time you take them. The best technologists rely on their knowledge, so they spend their time and efforts staying current. On the other hand, you have weak applicants who know a certification is their only way to a job so they put their efforts into it.

> Certs help with administrative things like HR requirements, contractual obligations, audits, etc.

You're pointing out things that are generally considered to be failures in our sphere anyway. I don't think that's unrelated.

HR brings us terrible candidates? Certs (currently) don't help that, and when HR over emphasizes them, we can blame the certs. (In theory, certs COULD help, but the people that get the cert instead of the experience are the problem ones, and those are the people HR brings us)

Contracts are an attempt to preset agreements between two parties, and in the tech world usually one of those parties are at a disadvantage in the tech space (otherwise we'd not be making the contract). Certs are used as a means of asserting competence in the employees, but the other party has every incentive to focus on the cert instead of the competence. So when it goes bad (as it often does), we can blame the certs as not achieving the goal.

Audits are likewise, trying to filter for competence (in action or in people, depending). But again, if the incentive is for the cert over the competence, it is the competence that suffers. We blame the cert (or more correctly, the emphasis on the cert).

IT definitely has a bias against certs. And it's not really against the certs themselves...it's that emphasis of the certs is harmful, not helpful. So we argue against that emphasis, which sounds a lot like hating the certs.

> The problem most IT people have is thinking that certs address technical issues, when in fact they address business issues.

But those business issues are "how do we ensure the technical issues are addressed?" And it turns out Certs in practice do a terrible job at that.

It sounds like when the poster said "address business issues" they meant "enhance regulatory/bureaucratic issues."

Nope, it is in fact the certifications. Strongly agree with the tptacek quote here.

So how would you approach scaling the IT Security industry without some form of industry certification process?

I'm definitely not trying to defend the CISSP/CEH style certs here but I don't see how you reliably expand the industry at scale without some form of certification process.

A company hiring it's first security person or a company trying to hire a lot of security people, need some form of base benchmark to work from, just like with most other professions (e.g. Law, accountancy, architecture etc)

What does "certification" have to do with scaling the industry? Training and nurturing talent is a hard problem, but expensive tests don't do anything to mitigate that problem.

Certifications are used by many industries to provide a demonstration of a common baseline level of knowledge and experience, so that each individual person doesn't need to be assessed by each hiring organisation.

For example Certified accountants, Lawyers etc.

Without some common baseline, how do people looking to hire security types who don't have the experience to assess their skills and knowledge avoid getting bad people?

Also the article's argument that "it's experience that counts" really doesn't help get new people into the industry, where do they get the experience in the first place, it's a catch-22

Apprenticeship works. Lots of places will have junior people work below senior ones on projects and gradually gain experience and become senior.

Yep I think apprenticeships can help too, there's no one thing that's going to help bring a load of people on, I think it's got to be multiple paths.

I believe that too. I simply believe --- with ample evidence --- that certifications aren't going to be one of those paths.

so, out of curiousity, that implies to me that you don't rate any IT security certifications?

So would I be right in thinking you don't think that any of the Offsec certs (OSCP/OCSE), CREST certs (CCT etc) or SANS certs are usful?

Also, and I'd be genuinely interested to hear your thoughts here, why do you think that IT/Info Sec will take a different path than other professions (medicine, law, accountancy, engineering etc) which fairly universally have evolved into a certified professional model?

The "certifications" in medicine and law accompany postgraduate degrees and are far, far more recognized than the random certificates you listed, some of which are profit-making enterprises from for-profit companies.

Ah ok, so would it be fair to say that you're not opposed to the concept of certifiation, per se, but that you're not a fan of existing options in the field?

Of course one problem is "how does a certification become recognized", I mean in IT security it's going to have to start somewhere...

In the UK the IISP are perhaps closest to the "traditional profession" certifications, but they're struggling a bit to get traction.

You can't wish professionalism into being. You have to build a profession. We're not there yet with any aspect of information security. The hard work of defining the field and its requirements has not yet been done. No organization currently extant on this planet has any business pretending that they know the answers to these questions, let alone charging money to take tests about them.

Obviously it takes time to build a profession, but you've got to start somewhere, and part of that path is certification.

Unfortunately the industry is growing far faster than perhaps happened for previous emergent professions, so the time needed to slowly grow professional bodies isn't available.

If it's not commercial organisations that start providing those services, the only other options I can see are some form of union, or some government mandated body. Those are options, but both have their challenges.

Both those options have their downsides.

No, you're describing a cart that is pulling its horse. The "certification", in whatever form it takes, must follow the professionalization of the field.

Regardless, none of the certificates you've mentioned --- OSCP, CREST, or SANS --- will define information security. None of them have any meaningful credibility to experts.

Interesting, so what's your view of how professionalization of the industry should get started?

The burden of proof should be on whoever is suggesting that security has anything at all to do with those professions, but I'll throw out a couple of observations anyway.

Those professions have rules, and are backed by either legislation or science. All participants are bound by said rules. For a lawyer, certain things are legal, certain things are not.

Security is a game where the whole objective is to either break the rules (and often the law) or to defend against someone who is.

How are you going to tell me that person A is qualified for the job based on his exam results, and person B is not, when person B got a root shell on your server and stole your data?

It's like the 1989 draft when the Giants tried to make Dion Sanders write an exam to see if he was qualified to play in the NFL.

"“They sat me down and gave me a thick book,” Sanders recalled. “I mean, this thing was thicker than a phone book. I said, ‘What’s this?’ They said, ‘This is our test that we give all the players.’ I said, ‘Excuse me, what pick do you have in the draft?’ They said, I think, 10th [actually 18th]. I said, ‘I’ll be gone before then. I’ll see y’all later. I ain’t got time for this.’ That’s a true story."

It's difficult to believe that anyone who can claim to really know computers thinks they aren't based on a series of interacting rules. That's basically all they are.

Understanding how those rules interact, how to trigger certain interactions others didn't intend, and the best practices to not get bit by those interactions is what security is all about. It's much like law or medicine in that you are looking at unexpected consequences of multiple complex systems interfering with one another and looking for compromises and best practices to keep the most disastrous interactions the least likely to happen.

Computers follows rules the same way a football player follows physics. Those are not the rules GP is talking about.

No. Computers are rules. They are multiple, complex, partially abstract and partially concrete rules systems with messy and often poorly defined interactions among those systems.

The individual NAND, AND, NOR, OR, and/or XOR logic? Rules - tabular even. The base IA? Rules. The microcode? Rules. The VMM? Rules. The TLB? Rules. The assembly? Rules. The OS kernel? Rules. The C library? Rules. The ABI calling convention? Rules. The application language? Rules - syntax and semantics. The libraries under the application - rules. The application itself is a list of rules for how data is processed. If it's Turing complete it's basically equivalent to the lamda calculus.

Every security issue is some misapplication of these rules due to someone not understanding the implications of the interactions of the rules. Every single one. Smashing the stack? It's applying a set a rules in a way the code author didn't anticipate. Overflowing a buffer? The code author didn't anticipate more data being stuffed in than the buffer was made to hold. Rowhammer? There are rules of semiconductor electronics interacting with the programming language, the IA, and the logic layout. SQL injection? Someone's applying the wrong rules to sanitize the input and someone else is giving input that takes advantage of the underlying rules of the programming language and the RDBMS that they were allowed to invoke because proper sanitization wasn't in place.

There is not a single person on this forum who needs your lecture about how, in one sense of the word, computers are based on rules. The obviousness of this fact should indicate that the disconnect is elsewhere.

The rules under discussion are policy, and the question is how to define policy for evaluating people's skill at running roughshod over policy. The fact that there are underlying "hard" rules is literally universal and therefore uninteresting in this context.

I really don't think you're following the point.

Lawyers and doctors deal in interacting complex systems of rules. So do information security people.

If you can make a certification that works for one expert in dealing with interacting complex systems of rules, you absolutely can make a certification for another expert in dealing with interacting complex systems of rules.

The details of what you test are different, but the fact that it's been done for law, medicine, medical specialties, dentistry, mechanical engineering, electrical engineering, civil engineering, and many other fact and rule based fields means it can most likely be done in general for people looking at how different systems of rules intersect.

No, you can't. That isn't how security works. Your offensive adversary follows no rules. They exist to break any rules you can think of. You can make all the rules you want and you can test people on their knowledge of them. Hackers do not care.

On the defense side, they simply do not work. Everybody gets hacked. The best companies with the biggest security budgets employing people at the cutting edge of security research still get hacked. Security experts get hacked. If the best in the industry still haven't solved this problem, you can't even begin to make the framework that you're proposing.

The discipline cannot be described as experts dealing with interacting complex systems of rules.

People still die. People still lose lawsuits. I-85 still partially collapsed. Yet people can be certified as knowing and following best practices in those fields.

Illness doesn't follow rules the doctor dictates. Rainfall and earthquakes aren't set by the engineering boards. How people adjust, prepare, and respond are where the rules humans can set work. Everything else is unchangeable rules.

In computers, as in some parts of law, we have ample opportunity to address underlying rules as well as the rules around how we adjust, prepare, and react.

The challenge that I've been trying to express is that individual excellence is hard to replicate at scale. Sure in sports, that works ok, the numbers are small and the rewards are great, the incentives are there for people to comb through thousands of people to find that one candidate who's truly excellent.

But that's not the reality for most companies, they're not trying to hire the absolute brightest and best, realistcally not everyone can. They're looking for some measurable indications that a candidate has a baseline level of knowledge in a given field.

Now I feel that a good certifiation can be part of that. So a valuable activity for the industry is to try and create better certifications to help companies who aren't in a position to judge for themselves whether someone is great at something or not, that there's a level of knowledge and understanding there.

If current certifications are poor, then it should be possible to articulate why they are poor, and describe what would make them better.

My references to other professions were designed to reference the fact that those professions have had to face similar issues as they've grown and in science, engineering, law, medicine, accountancy, etc etc they've pretty much all decided that some forms of professional certifications are the right way to go.

Not to say that they're perfect, but that they could be better than the alternatives.

How many of those industries does certification actually work in? Does it prevent dodgy lawyers and accountants?

Certification does work in some industries.

When a lawyer is admitted to the bar, that is generally taken as proof that they have some idea of what they are doing. Bar exams are hard.

And the bar also provides a forum for dealing with shady lawyers. If a lawyer treats you badly, you can file a complaint with their bar association and they might get disbarred. This is an area where cyber training and certs is not as good yet, I think: as a forum for resolving disputes.

TV may be leading me astray, but isn't disbarment usually for ethical reasons, not general incompetence?

The problem is one of trust, methinks. Most certs require only that you "know" the material well enough to pass a test that uses your memory. If the tests were empirical, say like the Red Hat tests or the Cisco CCIE, then the trust that someone actually has skills might be more believable.

Having worked in IT security for many years, I can attest to the fact that IT security is more of a subjective set of processes rather than a specifc product or set of products.

There are skills involved as far as tools and knowledge of how to use tools, but these change depending on the use case.

Yep the problem is that some of the well known certs are bad, not that the concept of IT security certifications are bad.

I've been in IT/Infosec for 17 years now. I have certs that I literally maintain as a HR/sales checkbox (e.g. CISSP) and certs that make me think every time I need to refresh them (e.g. CREST CCT). It's possible to have good IT sec. certs.

There's both some truth and some falsehood to the 'certifications don't prove anything' argument:

Answer a multiple choice test for an MCSE or whatever? Doesn't prove much.

Receive a server that's been wrecked and won't boot, turn it into a load balancing HTTPS server, SMTP server, a bunch of required cron jobs and a boat load more requirements for RHCE? Proves you can do those things.

Disclaimer: used to work at Red Hat. Still love their stuff. Cisco has task-based tests too.

Yeah it's just a shame that multiple choice tests are the standard rather than practical examinations. All the multiple choice tests look for is if you have memorized their documentation and can figure out the test format rather than if you can actually apply it, which are different skill sets.

I'd agree that the problem isn't so much that certification has to be bad, but that bad certifications (e.g. those that examine though purely multi-choice) aren't appropriate/good for the industry...

For me the answer is better certifications.

I just recently had someone ask me to take an assessment test for a senior developer position. There's always some silly hoop to jump through, so I thought "why not".

Well I got booted out of the test because I hit Ctrl-C to copy something for the first warning, and hit Ctrl-L (muscle memory) for the final revocation of the test.

I just thought to myself ... did I just fail an assessment test because I hit Ctrl-L?

The test gets reset and allows me to start back up, only it took 20 minutes off the allotted time for some reason. I didn't even get to the last question on the test.

The results showed me scoring in the 96 percentile. So I basically threw 3 questions away on this test and still scored that high. And I skipped one question because it was asking about building/creating msi files on an C#/ASP.Net/MVC assessment. I have no idea why it was there, but I don't regularly build msi executables (although I regularly automate them).

And the worst part is that a lot of the questions were inane things like "given this inheritance hierachy, sally adds the new keyword in front of one of the child methods, and then this other code uses this inheritance hierarchy. what is the output?".

At no point do I feel like anything on that assessment came even close to assessing my ability as a senior technical person. These were things a college student could have answered just as accurately.

I know it's not quite the same thing as a certification, but I seriously dislike assessment tests. Unfortunately it seems like every company has their games you have to play in order to actually get TO the technical folks.

I think I've done this test or a similar one. The worst was going for an MVC role and it was giving me a bunch of questions on webforms lifecycles and "what does the X.Y" method do.

I've had several of these in my job search recently as well, with a background in education I'm astonished that anyone thinks they show anything.

> There's no reason someone can't have both skills and certifications

Of course you're right that it's not impossible. But here's why it happens anyway and why the heuristic of them being roughly mutually exclusive is not insane:

1. There's a certification that's nearly meaningless because it's so easy to obtain without also having the relevant expertise that the certificate is supposed to represent.

2. People who are actually good at the thing will notice the certificate doesn't measure the skill correctly, and will also note that there are people in the world with this certificate who don't know the skill.

3. Those experts will not use the certificate when they hire people, and will not get the certificate since it doesn't work and no one who is an expert is using it to hire anyway.

4. Meanwhile, there are basically only two groups that care about the certification:

> a. People who are clueless: a clueless hiring manager who doesn't understand the domain they are hiring for so they are looking for cheap proxies for skill and experience. Clueless wannabe professionals who don't understand the domain or the industry enough to have been in the expert group above but who are still looking for jobs in the field. Clueless clients who are impressed by the certification because they don't know any better.

> b. The people who are taking advantage of the clueless clients: Professionals and hiring managers who know very well that the certification is worthless, but who use it for sales and marketing anyway because it mollifies clueless clients.

If that trend holds, then you have a signal (the certification) the repels experts, while attracting the clueless and those who would exploit them.

So when you find someone in the world with that certification, you should expect on average for them to be clueless or preying on the clueless. That's why treating it as mutually exclusive isn't insane heuristically, even though it's not impossible that someone who is good also has a certification.

Additionally technical certificates supposedly measure competency in a particular technology. That technology is obsolete roughly at the time you're sitting to write the exam. So you've spent time learning material (assuming you didn't already know it) to cover a technology that will be in the process of being replaced as you sit down to write the exam. You may as well have spent the time using the skills productively rather than using it to obtain a piece of paper that says your skills are obsolete...

Plus there's the financial outlay of it; costs to obtain the course material and sit the exam.

These points (including yours) basically amount to my bias against obtaining certificates.

I'm not against the material itself, if it's useful. But I'm not in the habit of paying for or spending time on education for anything other than the value of my interest in the material. A piece of obsolete paper does nothing for me at this point in my career.

YMMV

This is a great comment to illustrate grandparent's point that irrational hate of certifications permeates the industry. Many certifications under discussion here are meant to assist at the entry-level. That's what the certificate is "supposed to represent". That's what skill the certificates "measure correctly." That's the 3rd group of people who care: people operating at or near the entry level.

It is no surprise that experts hiring experts in any field couldn't care less about certifications (aside from ones required by regulation). At the expert level, you hire real, verifiable experience.

You've ignored the point of the post you're replying to. You're looking at the credential as a employee signalling tool, not a tool for other parties to satisfy a business need.

Your HR department needs avenues to sift through referrals and comparison points. If an individual has the certificate and compares equally with a non-certificate candidate, the first individual has signaled, through the certificate, that he is interested in the field, as well as willing to invest time and resources into advancing in that field. This is the flipside of the employee signalling.

If an applicant applies without the certificate to a post which requires it, he will recognize his cover letter/interview should make a point of demonstrating competence in the area of the certificate. This is an instance of employer signalling.

If an applicant fakes having acquired the certificate and there is an easy way to determine whether or not he is faking, then HR now has an easy sieve for removing employees willing to misrepresent their skills to obtain the job. This is a step in the HR QA process.

Some employers work outside the 'work for hire' states. If they contain a certificate requirement in a job posting with enumerated skills, they can point to ineptitude in the certificate skillset as reasons for dismissal depending on the jurisdiction they're in. This is a tool in legal's toolbox.

Perhaps the certificate system is the result of the collaboration of a number of employers attempting to pool credential and training requirements for an industry, then create requirements for credentialing into legislation to control the industry pipeline of workers. This is a method of signalling to legislators or restricting competitors by indirect means.

And so on.

Additionally, your fact pattern has a small problem: expert acting as hiring managers who have knowledge of the problems with the certificate are free to adjust their screening procedure to obtain more fully vetted candidates. The only time avoiding the certificate entirely is when the signal it provides is negative.

This doesn't mean that certs are useful, merely that they might be useful to stakeholders you aren't considering.

I did ignore that mostly, you're right. My basic claim is that on average this is actually true:

> The only time avoiding the certificate entirely is when the signal it provides is negative.

And further, I think all your examples are perfectly valid, real-world examples that I don't dispute exist and that lots of people find important. I also think perfectly good and reasonable people operate in the reality of their industries and play ball with these things when necessary. AND I think all the use cases you mentioned are bad for the system overall. As in, they are real, and in a practical sense we can't just ignore them, but ideally we wouldn't have them.

HR using negative signals for filtering is bad. Job requirements tailored to the actual job are good. Broad and mostly arbitrary requirements from a third party are bad. Applicants to a well specified job also know to address weaknesses in their cover letter. Specific and well-specified requirements are also useful in legal disputes. Using arbitrary requirements to provide legal for firing people is bad. Employers colluding to control the training pipeline using arbitrary requirements encoded in law is bad.

I agree with you that they useful in the real world, but I'm arguing that that usefulness is evidence that the system is worse than it could be.

> Your HR department needs avenues to sift through referrals and comparison points. If an individual has the certificate and compares equally with a non-certificate candidate, the first individual has signaled, through the certificate, that he is interested in the field, as well as willing to invest time and resources into advancing in that field. This is the flipside of the employee signalling.

The real root of the issue here is that HR can't do their job. An unintended side effect of all this signalling is that future employees will be glad to get filtered out because they don't want to work somewhere that hired on certification instead of competence.

> The only time avoiding the certificate entirely is when the signal it provides is negative.

It is possible for the same action send a positive signal when done by some people and a negative signal when done by others. Specifically there can be 'contersignaling' [0], which is basically signaling that you don't need to signal.

[0]: https://kelley.iu.edu/riharbau/cs-randfinal.pdf

   There's no reason someone can't have both skills and certifications

Ok, sure...But we're not saying they are mutually exclusive. We're talking about bayesian inference here...

Articles like this one frustrate me.

I'm 30, and am essentially starting life over after finishing my military enlistment a couple years ago. all the experience of setting up shops and drafting reports meant nothing with out a degree. So I start working on my degree, and I am absolutely miserable. My love of learning was sucked out of me because I wasn't learning: I was working towards an extra line on my resume.

Right now, I am in a jr. sysadmin position making minimum wage, but I was selected for a SANS scholarship where they pay for your GSEC, GCIH, and one elective cert. My friend bought me a decent laptop, so I could experiment on virtual machines. Another registered me for the NCL so I could access their gyms to spread my legs a bit with more powerful tools. I READ SECURITY WHITEPAPERS FOR FUN NOW. I love trying to figure out how to best balance company workflow and security best practices.

I know at the end, that the three certs are not going to make me a SME, but at the very least I hope that this particular extra line on my resume can help get my foot in the door somewhere I can be mentored and develop my base. A salary that can actually pay my bills would be nice too.

Then I read articles like this, and wonder if I'm going to be sidelined again. I feel like at that point, my life is worthless.

Don't fret. The author is correct, but it's mostly true for the upper ladder of the skilled workforce. I would chuckle if I saw a senior engineer list certifications on their resume (apart from maybe advanced CCNA/NP/IE cert for a networking specific role).

At that point in your career your experience and knowledge will show for itself, and you won't be proving anything with paper.

Right now, however, it's vital you get your foot in the door. You don't have much experience yet, so a cert shows you're eager and at least not totally clueless. The jobs that care about your certs will probably not be very good, but it's a stepping stone if you're ambitious.

This is exactly true for all non-work related qualifications (schooling, degrees, certification, etc)... they matter when you are first getting into the industry, and for your first job or two.

Once you get established, no one looks at those things.

What's so special about CCNA? I've acquired it as a backup plan few years earlier, but never used it nor worked in networking. Pretty much anyone with basic knowledge in networking, few weeks of spare time and and few hundreds of $ could obtain it.

The CCNA has value for network engineering positions.

To be honest, I think your original gut feeling is correct. Keep doing what you're doing. Especially for small companies, the hiring process is about finding someone who can adapt and push themselves. Seeing that you experimented and pursued formal certification means that your teachable and ambitions.

This article is talking from the perspective of getting hired by so-called "elite" security firms. That comprises a tiny percentage of the roles you might possibly seek in the future, and shouldn't taint your pursuit. Many employers who are trying to staff some sort of internal security competency will regard it very well -- there's a reason they appear in countless job listing -- as a sign of both focus and interest.

What, everyone doesn't aspire to become an elite hacker speaking at BlackHat?

Seriously, you are spot on. It takes years and dedication and no small amount of coincidence of interests and skills to reach the elite levels. It also takes a kind of persistence and thick skin to do the research and get the skills to get your first real high end job for most people. I tried replying to the OP about how to get to where our senior and principal consultants are and.. it turned into a somewhat muddy word bomb. At some level the advice was basically, "Yeah, just get really good at... everything, then infosec is easy"

There are so many paths and skillsets required and you can specialize in so many areas (operating systems, tools, crypto, memory corruption, etc...). How do you even begin to convey the depth and variety to someone at the start of their journey? Ultimately there are just a lot of common patterns of elite hackers, base skills you use all the time. Get those skills, and keep trying to hack stuff :)

I think you are doing it right. I'm also a veteran, I had no idea about certifications until I got out. It's more of a balance than this article represents. Certifications mean something, but they aren't the end-all-be-all that they bill themselves as. The trick for you is to combine the attitude towards work you hopefully developed in the military with this foundational knowledge to demonstrate 1. you are not an idiot and 2. you can get crap done. When I left the military I was making ID badges (with a master's degree). Keep at it, it gets better.

No one-thing will make or break your resume aside from the obvious egregious mistakes that I won't go into here.

Get your certifications - they will certainly help. Just don't pin all your hopes and dreams on these certifications.

The will add to your resume as a whole - so that when someone is reviewing your resume they can put an extra check mark in the pros column (experience, check; skill set, check; oh hey he has certs too, nice. check;)

In my experience working in technology, most people aren't as vocal about certifications as you would think based on the chatter you see online. They are a nice to have, not a must have, to get started. Sometimes depending on your actual job, they become a job requirement and work will pay for them.

Also in my experience, the people really against certs are people who for some reason don't like the idea of other people "invading their turf". As if you getting a cert in their field somehow trivializes their experience or effort.

In other words, keep doing what you're doing. View your certifications as milestones along the way, not the be-all-end-all of what you will need in your career. Continue learning and getting valuable experience, and you should be alright.

Update: One last note: certs are like a lot of other things you will encounter in any type of education. You get out of it what you put in. If you work just enough to be able to pass the tests, well, that's what you will get out of it; a way to pass the tests. If on the other hand, you try hard to learn and understand the concepts, then, that's what you will get out of it and it will certainly add to your learning.

You won't be sidelined. If you internalize most of the material from your SANS courses you'll probably be smarter than 2/3 of the people in this industry, if not more.

Most of the articles like this seem to come from people in the top 1-5%. Most of them are people that have started their own companies. I'm not a unicorn and most people aren't. I'm pretty confident that Tptacek and everyone else quoted are better security analysts than I am and possibly ever will be. However, I'm also confident that I'm pretty good at my job and I have the potential to get much better.

The important thing is to keep learning every single day.

I have no idea if you are or aren't (be careful about your assumptions!). But I am certain that certification has nothing to do with the delta between the two of us.

I'm not crediting my entire base of knowledge with a certification course, but I did learn quite a bit at some of the courses I've taken. I've also learned quite a bit from books, articles, security conference talks, and of course, by spending a ton of time putting the things I read/watch into practice.

I guess my point is that I agree with you that no one needs certifications, but I didn't think the contents of the courses I took were completely worthless.

IMO, some subfields of security are better suited to structured learning than others. For example, forensics can be taught very well in the format of a certification course. However, from my experience, exploitation and reverse engineering are pretty hard to learn in the same format.

I doubt the curricula of any certification is entirely worthless. You're saying you appreciate their value as a forcing function and as a set of guideposts for what to learn. I'm saying: there have to be cheaper ways of setting up forcing functions, and I know there are better guideposts on what to learn --- they're just not promoted as heavily as the certifications, because nobody (except hiring managers, who are too dumb to realize it) makes any money on them.

I agree completely. There are books that cover the same content in many cases, but not always. I've read quite a few of these books, sometimes they are actually better. I was fortunate enough to take all of my courses for free, but if I was paying $5k out of pocket each time I wouldn't recommend it. I think the norm is to have an employer pay for it.

I know that's true and I find that especially alarming, because it gives those employers a tremendous amount of leverage as gatekeepers to the industry (by underwriting certifications for people they elect to employ and retain).

There is nothing wrong with getting a cert. SANS offers some great courses. It's a good way for folks to start and get introduced into the industry. There are different roles in security. Soft skills are an important part of that, learning how to manage the process is just as important as being a specialist in Pen testing or Forensics, etc.

Don't get discouraged.

I switched careers at age 30 also (into programming). A buddy told me something that has held out to be true. Think of your career like you would guiding a canoe down a river. Parts of the river will bend and get narrow and parts will be wider and easier to navigate. Try to stick to the parts that are wider and easier to navigate and you'll be just fine. It's ok to add another line to your resume to allow you to get your foot in the door. After that it's up to you. Build up your own experience and that will shine through every time.

This guy is flat-out wrong. He bags on the CISSP - thats a friggin management cert, not a technical cert. Like bitching the CEH doesn't go hard into Opex/Capex.

Meanwhile on planet earth "draw up an inter-agency security agreement compliant with all local jurisdictional laws and industry regs" is also infosec and command line kung-fu will do fuck all to help you get it done.

This guy just drinks "unicorn" piss - he didn't get "trained", he's just so darn smart and hardworking and special. I bet his business card says "lead ninja" or some other IT fuckboy bullshit.

Exactly... CISSP shows that you have an understanding of risk, numerous compliance requirements, and how much basic housekeeping activities like asset inventory management or having proper data classification/access controls help in maintaining security. The title of "Information Systems Security Professional" suggests that you're knowledgeable enough to speak intelligently in all of the ten domains, but your everyday job might be in a single relatively non-technical domain, like "Business Continuity and Disaster Recovery Planning".

I wouldn't expect anyone with a CISSP to be an expert in "tech ninja" stuff, but he should be able to assess whether overall security is better served by investing in the "ninja work" or, for example, additional phishing training for employees, at a given point in time. This is certainly not a deficiency in CISSP, and I don't think anyone with enough experience in the infosec industry would have such an expectation.

I've been in the industry since 1995. I've worked for Fortune 500 companies. What's the experience I'm missing to appreciate the CISSP? Because from where I stand, it seems mostly like a scam to me.

Then by definition you don't have any expectations for "a CISSP to be an expert in 'tech ninja' stuff", as I was saying... ;-) I'll agree with you that, to an extent, all certifications are a scam, especially those with artificially high sit-down fees. My point is that, CISSP does not claim to be a gauge for whether you are a crypto expert, just that you should know the difference between basic types of encryption and when it makes sense to encrypt your company's data, so that an accountant in one of those Fortune 500 companies you mentioned doesn't make a costly mistake. In short, it's not about "how to trigger an RCE", but, if you're in an Ops role, about "how can I ensure my users are patched without delay, so that I can minimize the impact of an RCE". Does that make sense?

Roles I've held:

* ISP network security engineering

* Network penetration tester

* Software developer for network security products

* Application security assessor

* (Most recently) Security team lead

I've had these roles for small companies and for very large ones.

What experience am I missing that would lead me to change my mind about the CISSP? I don't think attempting to pigeonhole me as a "crypto expert" is going to persuade me, because that's not the span of my professional experience.

That's an impressive resume of roles, but security is more than just those areas.

I think the grandparent is trying to say that the CISSP is largely for non-technical security roles. People that manage large security organizations are generally believed to be the ones that benefit from the CISSP as they are not interested in the details and more on a 1000 foot strategic view.

Without knowing more details about the your specific expertise, I would say you probably haven't been in a role that would benefit from the CISSP by just looking at your list. If you've been the CISO for a large company with 400+ people reporting to you doing IS work, having a CISSP should at least help you prioritize the work that needs to be done. Likewise for many companies that have non-technical management in security organizations, a CISSP helps provide some background for them.

Have you actually looked at the CISSP material recently?

It's a hodge-podge of everything under the sun. The only thing it's able to prove is that

a) you have endurance and spare time to sit for a 4-6 hour multiple choice test

b) you can commit to rote memory a bunch of meaningless material which you are unlikely to encounter in real security/risk management role

It truly is the worst of the bunch, but for reasons yet explained, it's the defacto "must have" by bigCorps - which is why it gets picked on by so many folks: everyone knows it's bad, yet most people end up picking it up.

I haven't looked at it in years, but that hodge-podge of material was more than enough to provide an executive with the basics that they needed to know to manage an IS organization which IMO is the goal of the certificate. As others have mentioned, it is a management cert, not one for normal use.

There are plenty of worse certificates out there - I would argue that the CEH is probably the worst one at the moment (although they are making some changes to improve)

In 10+ years consulting for Fortune 100 companies, zero is the number I have seen with 400+ security staff. A 50 person security team is enormous even by the standards of financial services.

Well then your exposure is limited. Boeing's corporate information security organization has around that number, as do several of the other major defense contractors.

Yeah, but let's be realistic. There are very few technologists who are passionate about computing that would really enjoy compliance roles. Infosec is a huge banner and I am going to assume most hackers here are on the technical side. Also no cert can possibly prepare you for negotiating corporate IT security policy.

But even for us (a high end infosec consulting firm) knowing how to relay findings and risk concepts to executives can mean the difference in our work getting implemented, transforming an organization from average to above average in terms of how they approach information security.

Anyway, don't be such a cynic, we just run out of air when we get to the upper reaches of technology expertise so it makes is dumb :P

Sounds to me like you're doing the right kind of thing to break into the industry.

Whilst there are people that, unfortunately, take the attitude in the article, I think that there's a load of others that take a more balanced approach and recognise some of the value of certifications.

The other thing I'd recommend, if you're not already doing it, is get along to some of the chapter meetings and conferences that there are increasing numbers of in security.

In particular I'd recommend BSides conferences (http://www.securitybsides.com/w/page/12194156/FrontPage) there's loads of them around and they're good places to meet people in the industry and also in many cases the sponsors are looking to hire.

Hey. Again with these false dichotomies. The choice isn't between "certificates" and "never letting newcomers into the industry". In fact, if I accomplished one single thing at Matasano, it's getting newcomers onto our team.

https://sockpuppet.org/blog/2015/03/06/the-hiring-post/

I kind of resent my opposition to certification --- which I see principally as a way of keeping newcomers out of the industry, by requiring them to get expensive certificates to enter it --- being cast as opposition to new talent. I think opponents of certification are far, far more welcoming than the supporters are.

<sigh> it's not a false dichotomy. The comment I was replying to was specifically expressing disappointment that his efforts in getting certificate would be overlooked because of a negative attitude in the industry to those certifications. I was merely expressing encouragement that not everyone would look on those certification efforts negatively.

The article takes what I think to be an overly absolute position in suggesting that certifications are actually harmful to the industry.

I'm not suggesting that you are opposed to new talent, I've not said that anywhere.

What I've said is that I think that cerifications can be useful for newcomers in demonstrating effort/ability in a field.

I think that those certifications can be useful specifically in scaling entry to the industry (I'm not saying they need to be expensive, heck I'd love it if they were free, but someone has to pay for the effort required).

The problem with leaving individual companies to review every candidate from scratch is that it's a huge waste of effort. If you're starting a SOC and have to fill 50 spots and get 2000 CVs across your desk, you realistically are not going to be able to take an approach of manually interviewing every single candidate.

Now and I'm sure you know more than I , that doesn't apply to high-end security testing companies, but different types of roles require different approaches.

No, that's not all you said. Your original comment is right there for everyone to read. You attempted to co-opt a position on an orthogonal debate --- whether the industry is adequately welcoming to new talent --- as part of your position on certification. Since I'm a strong opponent of certification and I'm reasonably confident I've done more than you have to bring talent into this field, I object, vehemently, to that kind of rhetoric.

I'd appreciate it if you'd take a second to retract.

The original article it titled "Information Security Certifications are Worthless and Causing More Harm than Good"

yes?

The top comment expressed quite clearly discouragement that this attitude of negativity to certification would affect their job prospects.

Yes?

My comment line that I'm presuming you object to is

"Whilst there are people that, unfortunately, take the attitude in the article, I think that there's a load of others that take a more balanced approach and recognise some of the value of certifications."

Didn't mention you, wasn't intending to mention you, referred to the article which clearly takes the position that certifications are actively harmful to the industry, a position that I disagree with.

If you feel I've insulted you, I apologise for that, but I'm afraid I'm currently a bit unsure as to why you feel insulted.

Just in general, if you want to stay desirable, you'll always need to be taking the market's pulse. Ask real employers which certifications they value. But since you have to be able to actually do something and not just bluff well to work in this industry, everything hinges on skill at the core.

Focus on developing the skills, not the paper, even if the paper is pre-requisite to get promoted. Credentials should always be second priority. If you have the skills, you'll be in demand as long as this class of problems exists. People hire people to do something. Do that thing they want. Don't put your trust in any type of credential.

That said, very few people will hold worthless certificates against you, and risk-averse corporations will want to hire someone as highly decorated as possible so that they're clean if there's a lawsuit related to operator error or negligence. If they're available at low mental and financial cost, they won't hurt.

Don't get discouraged. Work on developing the skillset and the rest will flow. Get the certs as needed or as they're available, but do not attach your own sense of worth, value, or success to them. Your skills are what will distinguish you no matter how respected or despised your credentials become.

Keep your head up. I am a partner at a so called "boutique" firm. If you want to be elite and do the hard things in information security the certifications themselves will not do /much/ for you by themselves. You have to never stop learning and growing. A lot of people in corporate IT that maybe bump into the edges of real assessment work think the certs qualify them and they stop learning and growing and learning how to break software.

It comes down to a very simple concept. Can you make the computer do what you want? Can you find the flaws in its state machine and hack the shit out of it? Yes? Come join an elite firm. No? Go into corporate IT security or keep learning until you can take the raw machine code and make it do what you want.

What does running a bunch of tools have to do with that? Most certs are very tool focused. Some /might/ have you do some stuff that is more interesting and CTF like, but so what. It is still meant for mass certification. If you only study to exploit a buffer overflow or inject SQL you are missing the point (though those are valuable skills).

You need to fundamentally understand. You need to be able to model complex software architectures and understand all the complexity of a modern software architecture and ecosystem quickly. Why quickly? Because it changes quickly. Because there is often so much diversity and complexity for a security practitioner at our level that you have to change architectures seamlessly and at a high (not expert), but very high, level of proficiency. That means you have to write code, play with a diverse amount of modern software and programming languages and constantly be thinking about everything from the security perspective. Learn threat modeling. Learn software. Learn the low level bits of computers and the high level bits.

What does this mean? It means if you know all the command line switches for all the tools on Kali you won't ever get anywhere. You need to write code. You need to understand operating systems like a systems engineer. You need to know what is going on with hardware. Will you use it all every assessment? No. But it will inform and guide your choices and you will have the framework required to understand almost all software and hardware you come across.

We have been working hard on our work sample assessment in our hiring process for the last 9+ months. We have seen folks with an elite level of memory corruption (e.g. guys who find and write exploits for the DoD) experience do very poorly on assessment and we have seen 2nd year college kids get right to the heart of the sample and own it. We see a LOT of people who want to transition into infosec or work at a more hardcore level come in and throw every command on Kali at our work sample. (Amusingly you have to think and assess things, you don't need anything fancier than a hex editor and a programming language or two with their standard library). Does that mean someone good at memory corruption is bad at information security? Maybe. It means their skills are too narrow to assess and secure the typical systems our customers hire us for and we work on, and we work on a lot of important stuff.

So let's be more concrete:

* Get really good at Python or Ruby (Python is what we prefer, but Ruby is okay). Write code every day. Golang is fun and good too.

* Work through all of cryptopals until it hurts, read every paper you can along the way

* Take a couersera course on cryptography Dan Boneh's older one is nice -- you really need to understand the crypto primitives in modern use and how to use them safely, you don't need to know how to implement a side channel resistant AES or ChaCha, but you need to know when someone is screwing up with AES in CBC mode (they almost always are if they are using a crypto primitive)

* Build or contribute to some open source security tools

* Get really good with mitmproxy and or Burp so much stuff now is HTTPS and or WebSockets

* Know your web app LHF

* Read and understand OAUTH (do this later)

* Learn every common authorization model in existence and how authentication and authorization are /actually/ implemented

* Work through Micro corruption CTF, you will understand better how a computer works if you get through /every/ challenge

* Learn threat modeling (Shostack has nice writing about it)

* Find software. Break and threat model software. Find more software.

* Follow and, more importantly, endeavor to understand the work of prominent peoples that talk about BlackHat every year or build software people use (Bernstein, matthew green, and the charlie miller's of the world, understand their methodology first, walk through how they do things more than their results, don't be distracted by results, but the skills and effort they employed to get the results).

That is the basics. Get good at this and you can break most modern software. Then you can specialize. Along the way of doing this you will come across tons of interesting stuff and find places you want to investigate. This is just off the top of my head. This is the really hard thing about being really good... it takes time. You can't just wake up and decide to do this at a high level. Programming takes time. Learning crypto takes time. Learning HTTP takes time. Learning software stacks and modern software architecture takes time. At the end of the day this path is daunting and, like a sieve, it filters out all but the best technologists. Now you can imagine why the author may have taken the sort of down his nose view he did of certifications, because this is an immense and challenging thing.

Step back a bit and assume becoming elite at this is a 5-8 year journey, what do you do in the mean time? Write code every day. Work on only a few things at a time to ensure you can go deep enough and understand it. Do your certs, they give you great exposure to the variety of tech, but never stop at the level a cert gets you to if you want to progress. Figure out what you are enjoying right now and focus on that. You can feasibly get more entry level pen testing and assessment roles in corp security on the backbone of a few certs, getting good at programming and automating things, and going deep on a topic area that really interests you... web app testing is a great starter, but never settle for banging out LHF (Low Hanging Fruit) findings all day, learn how to build web apps, too.

You can also go more of a risk management and policy route. This requires you to have a breadth of knowledge, be deeper with at least a few things, and understand corporate security, but I swear, if you love technology and enjoy deep thinking these roles will suck the life out of you. They are where deep thought often goes to die, drowned by corporate policy. Anyhow, it is getting late. Good luck. Find my company and contact us, we will set you up on our work sample and you can see what it is like.

There's 'compliance security' and then there's 'street-smart security'. They are very different things.

Most organizations aim for compliance (it's cheap and easy). They base security on contracts, certs and insurance policies.

Street-smart security practitioners are appalled by this. And, management doesn't understand why the 'security people' aren't on-board with 'compliance'.

It's a lot like the old west with Cowboys and Indians. Two totally different world views.

Pretty sure you hit it on the head. Over time, security breeches should alleviate this gap.

IMHO, I dont think it will pan out this way. Companies are being breached, and neither their marketshare nor profits are being significantly affected. Aside from https://www.dailydot.com/layer8/code-spaces-hacked/ -- how many other companies do you know which really paid a significant price for being hacked?

Yes, Anthem/BCBS, Target, HD, Sony, etc, etc have all had losses.. but they really havent been long-term impacted it seems.

I dont know what the answer is, this sucks hard as both a consumer and an infosec person. I tend to view security as a "hidden performance" factor. As long as the security flaws don't inconvenience the paying customers too much, they simply don't care if they exist or not.

Compliances cover a lot of the basics.

Isn't compliance the reason we have "at least one upper and lower case letter, at least one number, at least one special character and a drop of blood from your first born" style passwords that ruin security?

You're not up to date on your compliance, this sort of stuff is forbidden by the last NIST regulation :D

So only 20 years until it filters down to us plebs then?

I was involved once in a criminal forensics case. The defense's "expert" witness was a one man computer shop. He had created his own "certifications" and listed them on his resumé as indications to the court of his suitability as a witness.

It was literally "person's-company-name Certified Forensic Examiner".

He had created about 6 certifications, all of which he held.

It's kinda funny, but also kinda scary that the court accepted this as proof of his qualifications. The prosecution never raised an objection to it either.

So he got the court to accept a self-signed cert as a trusted root.

I don't see how that's much different than asking someone to solemnly swear they are telling the truth, when most humans are as capable as lying about whether or not they are truthful as they are of lying about anything else.

If the court has no one capable of gauging the expertise of a witness, it has to trust in someone to do that for them, and if neither party in the case objects to the witness certifying himself as an expert, it has no reason to gainsay that assertion. It's really the prosecutor's failure alone, for letting that detail slip past.

Self appointed experts should not be accepted by courts. That is what caused cluster fuck of bite mark non-science and fire non-science.

The differente against lying is that you know when you lie. You dont know when you are overly confident but incompetent.

The court has plenty of people capable of gauging the expertise of the witness, just not at that level. Impeaching credibility is not the job of the court - it is the job of the opposing party.

If the other side of the case had an expert that didn't bring up the lack of credentials, or the litigators on the file didn't bother to do the most cursory of credibility impeachments, that's on them.

There is something very different about telling the truth and having knowledge.

Everyone can tell the truth, but not everyone knows technical details about something. He is using his certifications to show he knows technical details; he might even believe he knows those things, but he very well could be wrong.

Hell, he may even be right. He might be a real expert. He doesn't make his claim to know things any more credible by way of vouching for himself even if he's an actual expert.

Certifications sometimes set a terrible baseline, but at least it's an independent baseline.

I think requiring someone to swear to tell the truth is more a way to nail someone for perjury later than it is to make them tell the truth. It's like those immigration forms where they ask you if you have ever performed an act of terror (or somesuch). Nobody in their right mind is going to check the YES box, but if they find out later that you lied, it's much easier to charge you and jail or deport you.

Can/did the other side bring this up? I may be watching too much suits, but it seems like an easy way to ruin the credibility of the witness.

I think soon that this sentiment will start to apply to Universities. It seems inevitable at some point in the near future there will be an online 'university' (for lack of a better word) who's graduates will be considered equal or even better than a standard university education, particularly for tech related degrees.

Universities have been a centralized source of accreditation for a long time. All it takes is for someone to figure out how to restrict graduation and filter good candidates using testing or some other means to gain accreditation and acceptance of its graduates by industry.

That's exactly why a lot of bootcamps have come into existence, along with a guaranteed job in the industry at the end of it. Though many employers hire university grads as a sort of "signal" for people who can work hard, think critically, finish what they started etc. And I'm not saying one is better then the other, just noticing this trend of bootcamps popping up everywhere to replace university CS/CE education.

anecdotally most people I know who have gone through bootcamp have had major issues getting hired and the ones that did were hired into support/saleseng rather than software engineering.

I've seen the same as well, more so that most of them were taught one structured way to look at problems and only how to use specific tools, rather than why. There's definitely tradecraft learning necessary beyond a bootcamp.

> near future there will be an online 'university' who's graduates will be considered equal or even better than a standard university education

http://www.uoc.edu since 1994

Western Governors University is a choice in the US, too. I am currently attending, and it's different from any other college I have gone too. All of the classes are competency based and self-paced, meaning theoretically you can get a Bachelor's degree in 6 months.

It already has for me, I won't hold it against someone but it no longer indicates any basic level of knowledge.

This is based on my last batch of interns that had masters degrees but couldn't handle hello world. Their spoken english skills made it clear that they were completely incapable of understanding the lecturers.

Universities are a business, they are paid a lot to provide a piece of paper, so they provide it.

Certs show the candidate is in the upper 90% of the group. Its no different than fizzbuzz. Its very much like stack ranking and tossing out the bottom 10%, those being the cert-less.

My day job maybe 15, 20 years ago was basically Cisco CCNP Routing test. It was kinda useful to study and pass Cisco Switching test because switching is a different world of networking. Probably I was in the top 10% of router ops, but I was only in the top 90% of switch ops. For many jobs thats perfectly OK.

Something very few people like to talk about is self inflation of company requirements. Top 90th percentile is frankly more than good enough for most companies. Yes lots of self important strutting about rockstars and ninjas but all they really need, often all they can get, is top 90th percentile, and it works out fine.

A cert is not a Nobel prize or Congressional Medal of Honor. Its not even a PHD. Its kinda like graduating middle school, or having a clean-ish criminal record. Maybe the best example is its like passing a drug test for a job, having the self control to not get high for a whopping two or three days before the test is kind of a minimum display of self discipline to get a job.

> "Recruiter Thomas Ptacek, whose Chicago-based agency Starfighter specializes in recruiting security folk"

   NET::ERR_CERT_DATE_INVALID
   Subject: www.starfighters.io
   Issuer: Go Daddy Secure Certificate Authority - G2
   Expires on: Nov 13, 2016
   Current date: Apr 12, 2017

Is there some infosec version of Muphry's law?

Starfighter is defunct. https://twitter.com/tqbf/status/771533037666390017

Well that's less fun. They had just started too.

Was there ever a post detailing why? I searched for it several times but found nothing...

I think this will probably be it.

http://www.kalzumeus.com/2016/12/30/kalzumeus-software-year-...

My pet peeve with CISSP (and all other ISC^2 certifications) is this: https://www.isc2.org/candidate-background.aspx

It appears to say that if you ever hung out on IRC and tried to keep your handle private, you're ineligible.

That does sound pretty unrealistic, but doesn't this consider that scenario:

"Omit user identities or screen names with which you were publicly identified."

From most people I talk to, the exception is the OSCP since it requires you to actually pop real, live boxes. Anyone holding that cert has actually exploited a buffer overflow, escalated privileges, etc. CEH, CISSP, etc are just too theoretical with no hands-on requirements.

It's a joke as well, and it just means the holder could copy and paste an XP-era exploit, which has roughly no relevance today.

OSCP is a bit more than that to be fair, you're not going to pass with copy/paste of existing exploits.

Don't know if you have taken it in the last year or so since they updated it, but it's pretty tough. You may be able to use a public exploit to elevate your shell once on a box, but getting code execution was the difficult part. One of the challenges involved fuzzing, writing custom buffer overflow exploits, and dealing with weird stack pivots. That only got me about 20% of the way to passing the test. All in 24hrs. My girlfriend was taking the GPEN at the same time. While I was banging my head against a debugger she was making flash cards. I think that highlighted the difference between the certs.

Describe the overflow exploit you wrote. What was the vulnerability, and what did the exploit look like?

Unfortunately I can't get into too much detail because I had to sign an NDA (to prevent cheating). But the process was similar to when I have found them in the wild: identify the app, install it locally, fuzz various parameters (it was a real application, albeit an old one), find the crash, figure out stack space, figure out bad characters, find the right JMP ESP or equivalent instructions in a loaded library, write shell-code, encode shell-code, slap it all together, hope your hex math doesn't suck, run the exploit. No DEP, ASLR bypass, SEH manipulation, use after free, or heap related work - I learned that on my own.

Their web app challenges were fun too. LFI to code execution, SQL injection, things like that. They have a bunch of network related recon, standard red-teaming stuff.

The OSCE involves ASLR bypass, AV bypass, and using egg hunters.

The big thing about the OSCP, OSCE, OSEE certs is that you actually have to _do_ all of the stuff they teach you. Not a multiple choice or written question in sight. For the test they drop you in a network with vulnerable machines and you have 24, 48, and 72 hours (depending on the cert) to get code execution on each through various techniques. It was challenging, interesting, and satisfying.

Edit - it's worth mentioning that I still find vanilla buffer overflows on projects. These days most thick-client applications that I see are old as hell and are still vulnerable to exploitation techniques from decades ago. So while the skills that the cert makes you prove are cursory and introductory, they are still useful. In any case it's a good starting place for those that want to learn stuff on their own but do better when they are given the push to prove it.

In what way did the exploit they had you write differ from the kind we wrote in 1997?

It doesn't. The OSCE targets are Vista and 2K3 Server.

That being said, the nice thing about OSCP imo is that it gives you some structure and a well set up environment to play in. I think OSCP is a great entry-level certificate and serves as a good filter to interview junior candidates.

Does this help at a more elite level, nope, but that's also not the purpose of it.

This sounds like a 100-150 points (== entry-level) CTF challenge.

mona.py does that entire exploit in 1 command. It won't work against any supported version of Windows.

Having a 1-2 certifications on a specific domain means that we speak the "same language" regarding our work.

Red flag: someone that has an email signature with 50 letters next to his/her name,there is NO WAY someone is spent enough time on each: coding, security, audit, accountancy, at the age of 30 AND be proficient in all these domains.

>Having a 1-2 certifications on a specific domain means that we speak the "same language" regarding our work.

Or it means they can drop the same jargon and have maybe a passable understanding about the ideas that jargon is meant to convey.

Often people with extremely narrowly focused bases on knowledge (as indicated by having BAs, MAs, and a giant rap-sheet full of certs all in the same topic) have wound up being thoroughly incapable of actually applying any of their knowledge to the benefit of the team they're on because they just don't know how to get what's in their heads into the heads of people who weren't steeped in the same language as they were.

I run into this issue on my resume. Do i throw random skills i spent 4 months learning for some project and never used again? I feel like overloading these things devalues the skills i actually AM exceptionally competent at, as opposed to just capable.

Are they relevant to the position?

I'd keep them if they are and lose them otherwise.

My advice? Remember that your resume goes through at least 2 filters, HR and IT.

What happened was:

IT boss: "we need to hire a coder"

HR: "What skills?"

IT Boss: "Oh, Foo language. But if they're a good coder they can pick it up, so just a good coder"

HR: "...you're kidding, right? There are millions of resumes out there, most from people with no skills just trying to land a great job. Give me enough to filter"

IT Boss: (provides list of three things)

HR: "This still isn't enough. Practically EVERYONE will have these. Give me years of experience, skillsets, processes, etc.

IT Boss: "Fine, here" (gives long list of things that MAY be useful)

HR: (starts filtering resumes based on these words, removing lots of good people and including lots of bad people)

IT Boss: (looks at resumes) "These people are clearly lying and all over the place, I'm going to focus on one or two things to decide who to interview"

So in writing your resume, you want to make sure you have the buzzwords for the job to get past HR. These buzzwords are pretty much guaranteed to be on the job listing, even if they end up not being very essential to the job. Did they mention Scrum? Better have it on your resume, because you may be filtered out if it isn't, even if it's something you'd not consider worth listing. Also, use the same words. I once was asked if I had "shell experience", even though BASH was on my resume. I assume "Agile" and the various implementations are the same. If they mention XP, you better mention XP.

BUT when your resume then makes it to IT, who (1) know what these words mean and (2) aren't looking for the same things at all, you need to have what they want. I tend to use a sidebar on my resume to capture the HR buzzwords, and emphasize my work experience in the main body, so an IT person skimming it will see what they want to see.

One technique I've taken to handle HR buzzwords on things that I don't think are actually a big deal: If the job listing says "Must know React, Angular, Backbone, or other JS frameworks" and I wasn't really strong in any of them, I'd do enough research and testing coding to do a Hello World in them, then add "Exposure to Foo, Bar" on my resume. It tends to get through HR (word is present!), and I'm not lying to the IT people - they understand that I'm not claiming expertise, but I'm also saying I'm willing to give it a go.

As a corollary to all of this, you need to tweak your resume for every job posting, to match their buzzwords and remove ones they didn't list that aren't really core to your skills.

I find value in the SANS courses I've taken. Not hat the certs mean too much, since the tests are pretty easy... but the concepts, common jargon and best-practices discussions that take place can be useful.

It's kind of like college. It's about what you get out of it. From a hiring perspective, though - if a course provides utility, perhaps if one of the interviewers also has taken the cert course, they can probe the candidate more thoroughly on their knowledge from the course.

Nobody is knocking security training -- just the value of the certificate process.

We all know far too many people who have mastered the test, but not the material. This makes the signal from the certification unreliable.

Being a manager of an InfoSec team I agree with this, especially the CISSP and CEH.

I've seen a few folks get a CEH and then they're off to App testing land, but the funning thing is, none of them has ever written an app, some not even a script, and they are now doing security testing on mobile apps. Basically they just push a button on an app scanner and pull a report, it's sad.

The folks that do succeed in security are the ones with curiosity, experience and drive to learn.

As some one who is looking to get into security, I have that drive of why. At work I hate when the senior engineers close something without explaining it.

I like to know how something broke and why it broke. I understand programming and can read about any normal language to a basic degree and lightly troubleshoot.

Your absolutely right, about those kinds of people too. Some get the certification and stop there. Others get it and use it as a foundation and build on it.

It doesn't seem like the certs themselves are causing more harm than good, it seems like the lack of evaluation employers do for potential employees is the real issue. Giving too much credence to essentially any degree is problematic, I don't think security certs are exceptional, in this case.

I joined the workforce at the same time that people were realizing that certs were a joke. That both hurt and helped me, as people hired me based on talent, and not a piece of paper.

But I wasn't in management, and I've since learned how very little technical skill you actually need to be an effective manager, and now I realize that certs are a great way for a manager to understand a complete baseline of the concepts needed for a particular field.

A manager does not need to be a hacker, but they need to understand a baseline of security concepts. That's what certs are really useful for.

I think the point is more that security certifications CAN be worthless and you don't NEED them, but that doesn't make them inherently bad/worthless. I think the author's argument should be that the industry has begun to rely on them too heavily for vetting. That makes sense though because it can be very difficult to vet the skills of a client. The hiring process is very time consuming so if you see two candidates and one has "proven" they at least have some baseline skill in an area then they will lean on that for decision making in the same way they look at education or self reported experience.

Experience on a resume is self reported so that is an even worse indicator of skill than a cert. At least one of those two involved external validation by a 3rd party.

I think there are a few good ones out there and getting them ensure the person has at least a baseline knowledge of some subject. I have worked in the industry for years as a pentester, but I still went and got my OSCP and OSCE for fun. A lot of it was review, but it was nice to fill in some gaps and practice things I hadn't had as much experience with.

Certs are like college degrees, you can get by without them, but it can be easier if you have them. You will probably learn some things along the way and the provide a foundation for later studying or pursuit. You don't NEED them, but you don't need a lot of things in life, that doesn't make them worthless.

So the article alluded to reading books and hacking on your own. But for those who need some sort of curriculum, progress bar, or structure, what would HN recommend to get to some sort of level of competency in the infosec field (like intermediate level/beginner-advanced).

Others will likely have more informed opinions, but here's some stuff:

Book: Web Application Hacker Handbook http://www.wiley.com/WileyCDA/WileyTitle/productCd-111802647...

I've seen it highly recommended and if you're not familiar with the field it's a good overview of exploit types for web apps.

Online training for free or cheap: Cybrary - mostly okay, but free.

PluralSight - https://www.pluralsight.com/browse/it-ops/security

Coursera has a Cybersecurity Fundamentals specializationd that's pretty good - https://www.coursera.org/specializations/cyber-security

Other books, if you wanted to go down the reverse engineering route:

Assembly Language Step-by-Step: Programming with Linux

The IDA Pro Book (for the strangely hard to buy IDA Pro, but the free version is pretty good)

Practical Malware Analysis

Bear in mind that IT security goes far beyond something with a processor in it.

There are physical access controls, personnel assessments, probability and impact assessments, budgeting, people-monitoring, process analysis and modelling...

Computers are a tiny part of it. This being HN I have understanding for the bias though.

Thank you!

Many certifications are worthless, but I would argue that not all of them are.

I learned more useful, practical concepts and skills by taking a couple of SANS courses than I did in four years of a CIS program at a University. Both my university classes and the SANS courses consisted of books, presentations, lectures, and individual or group assignments/labs. If they are taught the same as a university course, why is it automatically considered inferior? SANS teaches a lot of tools, but they also teach the underlying concepts to prevent people from becoming dependent on tools. In some subjects, it would be insane to not teach tools. For example, I took the GREM (Malware Analysis) course. Its an very basic course, but it would be foolish for anyone to teach a course about reverse engineering or malware analysis without using IDA or OllyDBG.

While the class won't (and doesn't claim to) turn someone into a professional-level reverse engineer, this course helped me understand a few things about assembly that I just wasn't comprehending when I used other sources.

Would I attempt to use my GREM as justification for applying to a malware analysis job? Of course not, but the course has helped put me on the right path. Its possible many people learned through another, far less expensive method, but that doesn't mean the training was worthless.

If I were hiring someone, I wouldn't use certifications as a sign that they are qualified. However, I would use the certifications listed on their resumes, combined with their work experience, to figure out what kinds of questions I should ask them.

I also wouldn't use certifications or a lack of certifications to disqualify a person. Using your anecdotal evidence of bad experiences with certified people to label all of them as incompetent is ignorant.

I don't really go along with this post. I think the title is a good eye-catcher, but inaccurate. Terry seems biased toward working on the cutting edge of embedded firmware and I agree that is really important work. I also agree that a certification like the CISSP (I have one of those) won't prepare you for that sort of work.

I'll wildly speculate that some potential Tactical Network Solutions customers are asking about DoD 8570.1 security certifications. That may be a mistake on their part for something as far down in the weeds as embedded firmware solutions or the Centrifuge IoT Security Platform.

On the other hand, some of Terry's customers likely hold those 8570.1 certifications, so he might want to be careful about rubbing in how they wasted their time in acquiring them.

If he takes a look at SI-7 in the System and Information Integrity control family (found in NIST Special Publication 800-53) he might find some selling points for his products. A certified security engineer who had done a RMF audit would know that ;)

@tptacek and everyone else,

What are your opinions on colleges/Universities with degrees focusing on Cybersecurity?

Such as Utica and there Bachelor/Master degree in Cybersecurity. [1][2]

[1] Program info: http://programs.online.utica.edu/programs/online-cyber-secur...

Curriculum: http://programs.online.utica.edu/programs/bachelor-cyber-sec...

[2] Program info: http://programs.online.utica.edu/programs/masters-cybersecur...

I feel this way about certifications in general. I work with tons of Microsoft and Cisco certified people and the basic computer science errors they make has be doubting the value of those certifications.

Simul-post :-)

The problem is, you can't tell the difference between the person that did the time, studied, did labs, etc... And the one that downloaded a test answer brain dump and memorized the answers.

And even in the case of the former, you can't cram 20 years of knowledge and experience into a certification.

I have just an MCITP that I had to get as a job requirement. My employer paid for a 2-week bootcamp. I did very little studying and already knew 90% of what was covered.

I have about 10 people around me that have various combinations of certifications and degrees that all use me as a lifeline when they get stumped.

Why would cisco people need to know CS? Different fields.

I mean simple mistakes, like not understanding that information on an air gapped machine is unknowable to another machine on the network.

"If you took the tax id and social from 'airGappedMachine'"

"It has no connection to any other machine"

"Just query the database"

"Store it on a thumbdrive and walk it over? It changes quite often I don't think that's a good workflow"

"No, just query it"

"How?"

"SQL!"

Or the classic:

"That password only has N characters, I can crack that in Y seconds"

"You are the Administrator of that box, you can simply reset the password and do whatever you like."

"Yeah but the hackers.... Y seconds"

"It's not Y seconds for them, if they already are an Administrator or have read database access it's game over anyway they have to use the API and that locks you out after 3 attempts. Also your are assuming a much easier hashing method than is actually in use."

"Yeah but my calculator says Y seconds so it's Y seconds"

Classic. This could be the script for one of those animated videos that https://www.youtube.com/user/gar1t did.

Like the "Mongodb is web scale" one.

I had this comment from a recruiter on LinkedIn:

    The CCNA Security only costs $180. It's a very small investment and I don't see why anyone serious about security wouldn't spend the money to get certified

It's like they viewed it as being completely outside of any actual training and development time - even the person hiring saw it as a sticker you were meant to buy.

Edit: Interestingly, this vendor specific cert is far more valuable in my country than anything like CEH if recruiters are anything to go by

“Would you feel comfortable letting a doctor be your primary care physician if all it took was to pass a written multiple choice exam?”

Doctors have to go through extensive certification in order to be hired, and constantly have to re-certify. The difference, though, is that a doctor's certification is very rigorous and well-designed.

The difference is that a medical certification proves competency. Certifications in our field do not.

Why is the medical industry able to make decent tests of competency but devs/security groups cannot?

Absence of them when you're a consultant is the issue. Its not that it wins you clients by having them, but not having them might lose you opportunities. Also, not obtaining them (especially if you know what you're doing) shows either potential laziness or "better than everyone" attitude that also is negative. The thrust of that article was exceptionally tilted to that attitude, and I would think twice about hiring someone with such a huge head. I'm sure those guys are good at what they do. If so, take the minimal effort to credential yourself so when people see your name on paper, they have some reference that you know what you're doing. Without it, is the readers imagination.

I have CEH, EnCE, and EnCEP. Doing CISSP this year. Why? Because it makes me stand out regardless. And I've landed clients who were amused by the "ethical hacker" destination. So don't undervalue cert just because of some cocky nerds.

It depends on the market you're after. Sophisticated buyers won't ask for certifications. They can look at your work and understand your value. Those are the clients my firm is after. Leads that ask for certifications drop out of our sales process, and I refer them elsewhere.

Part of the problem with certifications is that lots of students look at them as a means to an end. This is wrong and counterproductive. Learning to pass a certification is the laziness, most counterproductive exercise you can do to learn security. Yet this is common. Learn by doing. Then get a cert if someone demands it or offers you more money for one.

If more people approached certification that way, there would be less industry-wide pushback about it.

Sure, but who are you standing out from? Do you want to optimize your career to stand out from those candidates, or from the person you've optimized yourself to be currently?

Stated another way, getting a bunch of certifications helps you stand out from the entry level. Putting aside a platonic ideal of what certifications should be, you could have done that without those certifications.

With regards to your first paragraph - at this point my consultancy bills five figures per week, and I do absolutely no outbound lead gen. The lack of certifications will lose you business, yes. But I would argue that was not necessarily business you wanted to optimize for.

There is a similar issue with technology certifications (e.g. FIPS 140-2).

A lot of companies treat these as some kind of mystical incantations that will protect them if sufficiently invoked. Case in point: being mandated to switch from one OTP generator app to another because the latter is "FIPS-Compliant" - regardless of the fact that both generate the exact same set of OTPs.

This cargo-culting is not inherently harmful, but it leads to magical thinking and a false senses of security, as well as diverting time and energy away from more productive avenues.

I suspect that the CISSP-genre of certifications suffers from a similar pathology: intrinsically they do function as at least a partial indicator of some type of competence. The problem is when actors with a financial incentive to game the system meet up with bureaucracies: the less defined but more accurate metrics are thrown under the bus in favour of something that is easy to quantify and sell.

Every comment seems to be about whether the certs demonstrate anything but this article says, "a job description requiring a CISSP was a warning flag to industry elite not to apply."

In other words: if the company asks for certs it's the equivalent of wanting "6 years of react.js experience". I completely agree.

I am familiar with people who have purchased undergrad and grad level degrees in various fields as well.

The reality is recognizing the importance of a foundation of education is critical. there will always be shortcuts that people take in every imaginable part of life. With that said, people who have a firm education or knowledge no matter where it is from (institution and/or self-taught) will be able to point out people who took short ccuts fairly quicky. The challenge is knowing the right course of action to take, firing or other knee-jerk reactions can result in more harm than good in some situations.

I had a CISSP certification, I let it expired, I couldn't afford traveling and going to conferences to get the Continuous Education points.

If you do the math, attending webminars, reading books and writing reviews don't get you all the points you need every year (I can't imagine the fraud that must be going on for people trying to get those points). So I said "fuck it".

Funny thing is, I'm way more experience in security now (with CISSP expired due to stupid points) than when I got it and was certified. Joke and money-grabbing scheme no doubt.

It's the oldest story in tech, certs are "worthless" but look at almost any infosec job posting and you'll see:

   Ideal candidate will have CISSP, OSCP, CEH, SSCP, WTFBBQ, etc etc

That is because most of these job postings are written by HR who have no other tool to filter successful candidates.

Oh, I know, it's a sick loop. I guess the only way to play is to have two versions of your resume, one with the certs and one without.

Btw, I conducted pentests for over a hundred of companies (sakurity.com), and none of them ever mentioned any certifications. Maybe 1% of deals failed because of government-specific clearance.

I heard this back in the day when the CNE was "money printing machine, " and the MCSE just kicked off. So nothing has changed from the complainer side of things.

Certs have value depending on the person's skills. Never hire just because they have a CISSP or XYZ.

Sometimes it's just the key to the door to the interview. From there it's up to Employer to properly vet the candidate. If they hire an idiot with a paper cert, then it's their fault... and then they write an article. LOL

My experience with all the certified security consultants is that they refuse to put any effort to understand the end risk of a security vulnerability on the product that is being audited. For e.g. for many sites, it doesnt make sense to implement security features that are required by online banks.

As an aside question for web developers: How many web developers encrypt/checksum all fields on the client-side?

I wanted to get my consulting company into PCI auditing and you need certifications to do that. One problem with the certifications is they aren't actually skill based.

I wouldn't be able to get one despite having experience because:

For several of them it requires years of work experience with a specific job titles (your job needs to be security, it can't just be part of your job) and the continuing education credits are expensive and largely not helpful.

I would love if there was a recognized certification that was actually interested in proving your skills not for making money.

I know other disciplines are the same way.

They don't really serve to keep unskilled people out, they serve as kind of an elite paywall where you need time and money to break through. And breaking through is largely an exercise in brute force not in skill.

"I wanted to get my consulting company into PCI auditing and you need certifications to do that. One problem with the certifications is they aren't actually skill based."

http://www.rsync.net/resources/regulatory/pci.html

I am wondering how many of these wise commenters have actually taken the CISSP exam, or even know what it is? It costs $600 and only has a 60% pass rate even after it requires certain validations in order to achieve a test appointment.

In discussions like these I really wish people would post up front if they had taken the test (regardless of pass or fail). This lets me know which comments I can ignore as completely ignorant.

I once tried to hire for a MS DBA position. The certified MCDBA's could not tell me what an index was or how it would be used. I was shocked so I went and took the tests myself. There were a few questions on indexes but I could see how someone could answer those questions and forget the exam cram by the next week. We ended up hiring someone with no certs and no degree and he was fantastic at the job.

Ah, yes... the "paper tiger". While I do have certs myself, it's only because my employer has required them. Left to my own devices, I would never bother. I've been in IT for almost 20 years and I agree that the "no cert/no degree" guys usually work out.

The team lead whose feet I studied at in my first few years in IT had a high school diploma and he could run rings around the guys from Carnegie Mellon and RIT that worked with us. He was a deep diver mentally. I watched this guy drop awk, sed, bash strings a mile long, write Perl scripts without consulting a single web page. He knew iOS (Cisco), Perl, TCL/Tk, Sun, BSD, and about everything else. He could configure a HA UNIX servers and Oracle DB backends without consulting a manual. It was most impressive. He was let go because he was a team lead (middle manager). The people that replaced him--yes, people-- knew nothing in comparison.

I'd venture to say most certifications in general are junk, CYA processes. If you are basing your hiring decision on the letters someone puts after their name, you're doing it wrong.

The same could probably be said for a degree, particularly with the current snowflake "everyone must pass" mentality at so many schools.

Certifications and degrees are nice, but neither guarantees a person is competent in any field.

I'm going to partially disagree with the article. the problem with the approach of "just learn to be a good security person" is that it doesn't scale. Sure back when I, and a lot of other people who are a bit older, learned security that was the only option, there weren't structured courses and certifications.

However when we're working at scale, certifications can be useful as providing a demonstration that the holder has some level of exposure/knowledge of the arena in question.

what complicatates this quite a bit is that some of the more popular certifications are, rightly, not considered that good as the process to get them lends itself to rote learning. So things like the CEH and CISSP where the exam is multi-choice, not so great.

On the other hand things like the CREST CCT definitely require a decent level of knowledge to pass and you need to be able to apply that knowledge in a practical situation and in time limited conditions.

I find the anti-certification bias in IT and security a bit odd really. If you look at other professions (e.g. law, accountancy, architecture etc etc) it's recognised that these things are required to get a minimum level in place in situations where you have a large body of people, I don't really see why IT Security should be any different.

To me, the problem of "these certifications are bad" should have an answer of "lets make better certifications" not "lets not use certification"

The only certifications I've picked up so far along my industry journey (I have no college degree and don't plan to get one, so these are necessary) are the Redhat RCSA and RHCE. Both of these certs can't be solved with rote memorization, and required me to log into a virtual machine and solve problems in a live environment, running through a plethora of common systems administration tasks that were then graded by how well I accomplished the requirements. Often the requirements were vague enough that I had to do some digging.

There was no real memorization needed, and I had the full man pages of the operating system at my fingertips, but the time limit ensured that I needed to have at least a certain degree of proficiency with each tool to finish all of my tasks before the end of the exam.

I assumed this was the norm with certifications, but the comments here strongly suggest that it is not. Are Security Certifications really multiple choice questions without any practical applications? That seems like it could stand to be improved greatly.

some are bad certs (rote learning, multiple choice), others are not (they have good practical elements).

One problem is that some of the "bad certs" e.g. CEH, CISSP , are well established.

My feeling is that the answer isn't "don't have certs" but "have better certs"

You're presenting a false dichotomy. The choice isn't between "security certification" and "people learning on their own".

the original article, I felt, made that point. It provided several examples of people learning on their own and provided those as an argument for why certifications were unecessary.

So not my dichotomy.

A lot of the bigger certs have all kinds of high-dollar prerequisites and accounting that only larger firms are going to bankroll. Perhaps the certs aren't so much an expression of how "good" you are as much as they're a token of how much faith "the man" has that you will stick around long enough to recover the investment.

Disagree with the sensationalism of the article but for anicdata: I interviewed 41 security engineers last year. Gave offers to 8, hired 5.

A few, less than 10, had certs listed on their resumes and no one in the interview loops gave weight to those certs or even talked about them from what I recall

True infosec l33t$ know the only real certification is being able to recite from memory the lyrics of https://www.youtube.com/watch?v=FoUWHfh733Y

Certs value, if any, are to show a very basic skill level, or starting point. That's it. If you are hiring someone just because they have a CISSP where another individual does not, you don't know what you are doing either.

While I agree that many of the certifications are worthless they do help newcomers to get into the security field.

I was lucky enough to start early when the only thing you had to do was to show your skills and willingness to learn. Those days are long gone but you can still prove yourself by delivering awesome security research or commentary - and people will hire you based on that.

That being said, any IT field which requires some sort "technical" certification to get hired is probably not the field you would like to get into. Why? Too much competition and race for the bottom line.

Luckily this is not exactly the case with Information Security - yet. In our line of work there are people who do a lot of the uplifting where certifications do matter. I would not say these are very technical jobs and they are totally dispensable. Other professionals acquire specific skill sets which are rare or difficult to obtain and as a result they tend to have the upper hand. There is of course a lot in between.

I do not mean that everyone who has certifications next to their name sucks. Not at all. But unfortunately, unless you are applying for a commoditized IT security field or security manager type of role, it will raise some questions.

Keep in mind that HR is also partially to blame. Many people do not know how internal HR teams typically work which is essential to understand why certain things happen the way they do when hired or when progressing through the ranks of a company.

HR are typically not technical and they are not experts in security either so they do not know what exactly they should be looking for. Of course they are not completely clueless but a seasoned hacker can smell bullshit a far while a well informed HR cannot. HR's filter is your CV and the job spec and these two are simply not enough to evaluate a successful candidate. Some job specs are written by HR themselves which is crazy because they are not in the position to formulate the role so they have to stick to what is known - i.e. certifications.

That being said there is a shortage of IT security professionals. As a result of that almost anyone can get hired if they show the right attributes. It does not take a long time.

The only thing that bothers me with the InfoSec industry these days are not the certs but that companies tend to have really bad hires (maybe due to bad certifications) who due to lack of deep understandings of the subject work on things that practically do not matter. As a result of that these companies have the illusion that they are doing something while the fact is that they do not in a practical sense. This is why in many places no one knows what the security department does - I am not kidding.

"If you must obtain a security certificate for compliance or regulator reasons, so be it."

It's interesting that his argument is against certs for companies but then he looks at regulation as unchangeable.

If I'm a competent enough services and web developer wanting to move into infosec, what else could I be doing to get my foot in the door besides collecting certs, as ostensibly shite as they are?

Bug bounties. Andddd that's it, you're done. Find a few in recognizable companies, and jobs will simply come to you.

I'm not going to engage in the debate about what certifications should be in the industry, but I'm happy to show which option is most advantageous for your particular needs right now:

* Certifications mostly do not teach you anything that you, as a competent web developer, cannot learn from the same five textbooks tptacek, others and myself recommend in these threads.

* Certifications cost money.

* Certifications optimize for companies and roles that disproportionately do not pay highly.

* Many certifications require upkeep.

Let's contrast with bug bounties:

* Bug bounties grow your real-world, hands on experience.

* Bug bounties do not cost you anything (in fact, you can get paid!).

* Bug bounties cover a much more diverse and up to date set of security flaws than certifications.

* Bug bounties optimize for companies and roles that will respect you more highly, pay you far better and aggressively try to hire you after you find more than, let's say, two serious vulnerabilities in recognizable companies.

* Bug bounty recognitions do not expire.

This is the best hot take I've seen on how to deal with certifications in security so far. This is an excellent suggestion.

so, How does one get into security? (to be specific like Incident response and analyzing intrusions)

There is a lot of self promotion and all my friends agree with me type logic in this article.

OSCP for the win.

Reply on news.ycombinator.com