I also don't think ctf's are a good model for security questions during interviews. Knowing how to do something securely is not the same as knowing how to exploit a vuln.
Reversing a binary tree on a whiteboard is certainly a bad question to ask. But I would argue, for all intends and purposes still a miles better indicator about future potential than if someone knows how/why TLS work. Yeah you can read that in a book. I can google it. Useless for interviews.
If you are hiring for a position that requires you to implement TLS, sure go for it. But that is not the rule. And what are you going to do after he has implemented TLS? Will he be able to work on something completely different?
So, in other words, exactly like algorithms then?
(Admittedly, candidates are likely to have memorized the algorithm for reversing a binary tree since that is such a common interview question)
Exactly my point. Testing for "how to implement a known algorithm" is much the same as recalling facts about TLS. You're testing recall only.
If you are the interviewer, even if you are asking a standard "how to invert a binary tree" type algorithm question, you hold the cards to keep pushing the bounds for problem solving by extending the question.
If you hire based on knowing how the the internet works (TLS, HTTP, BGP, whatever), then you'll be working with a bunch of people that understand how the internet works.
I know which team I'd rather join.
Now, if I'm hiring a sysop / devop / security engineer, it's going to be differently focused to some extent, but the same principles apply - core knowledge, communication, humility, ability to research.
i though like you before doing interviews
Some of the absolute best engineers that I've worked with take their time to wrap their heads completely around a problem before diving in. They aren't slow thinkers, but they aren't people who excel at these kinds of interviews either.
I've been doing interviews for a long time now and I find it more effective to surface strong opinions about things they've worked on -- good and bad.
I'm not hiring into a feature factory -- I don't care about fast cogs. I'm hiring people who care about what they do and giving them an environment to thrive in.
Meanwhile there's a decent chance of them running into a problem that looks like binary tree manipulation.
Despite all the handwringing about btree reversal, not being able to improvise a solution to that is a much more useful indicator than not being able to describe even the basics of TLS.
A security engineer is just as likely to derive a tls cipher config from first principles instead of googling/looking at ssllabs as a programmer is likely to reverse a binary tree from scratch instead of using a library or searching stack overflow.
Engineers (of any stripe) aren't hired to recite random knowledge, they're hired to know what knowledge is appropriate to apply to a particular situation.
It's probably not even that hard, but I doubt I'd be able to improvise a solution in an interview setting that's nothing like a real work environment. (Memories of trying to work out some kind of graph traversal while someone was basically just staring at me.)
It seems to me that people who are doing this kind of work behind the scenes of a web app aren't really doing web development.
As someone in a public company with actual customers, we have to deal with TLS configuration all the time (as customers come with requirements and being public comes with compliance/security) and it's important to know what we're doing there...
I'm responsible for millions of dollars in infrastructure and the way that I got here was being a web developer who knows how the internet works. And in an engineering organization with hundreds of engineers, most of them tend to come to me first with questions.
I have never once in my career had to reverse a btree.
Security vulns move waaay faster than algos
Setting your security test labs takes way more effort than opening IDE, LeetCode, checking informatic olympics tasks or maybe some book/pdf/write up/wiki
When developing algos you're in your own world meanwhile security often has to mess with other things like software, standards and stuff
e.g you're interested in web sec/hacking, then except understanding of standards (what it allows and what not, etc.) then you have various implementations to care about like web browser - chromium, gecko, ie, safari and stuff. It's a lot of effort!
Let's say that you want to find vulns in PDF parsers/renderers by checking their code source code - I think it'd take a lot of effort to check and understand (let alone exploit) those implementations in two major browsers (I suppose they're different, but I've never checked that)
>I also don't think ctf's are a good model for security questions during interviews
I didn't meant that, sorry if I made it sound as if I expected people to solve CTF tasks during SE interviews
Just wanted to encourage people to do cool stuff :)
It depends a lot on what you're doing of course. I do web application security stuff (glorified xss detector), i've personally felt that the most useful tools by far are the browser dev console and curl, but ymmv.
> I didn't meant that, sorry if I made it sound as if I expected people to solve CTF tasks during SE interviews
>Just wanted to encourage people to do cool stuff :)
Oh i definitely agree, ctfs can be a lot of fun.
Most compromises are credential stuffing.