Hacker News Reader

Comment by tptacek | Hacker News Reader

The best, most successful software pentesting teams barely market at all. NCC is one of the largest in the US, and nobody is finding them through Google ads, or, really, ads anywhere. The same is true off firms like Bishop Fox, Leviathan, and IOActive.

Which is to say, at least in the app pentesting market, I'm a little skeptical of the premise.

View on HN

The complaints I have heard is similar to the big 4.

Many go to them, they see awesome resumes. They also see very large costs. The the customer finds he doesn't get the A-team, but the F-Team, due to 'unprecedented demand'.

This is about making the process transparent.

That's a line every consultancy that ever had to compete with another consultancy has told at least one client. But the truth is, for the overwhelming majority of application software, and particularly for any software that would procure a pentest through a market like this, the Bishop Fox "F-Team" is perfectly up to the task and far more reliable than a talented rando.

The software pentesters with gold-plated resumes do high-value targets (because there are more high-value targets than there are pentesters to service them). Google is not going to source Google Mail pentesters on DICE. Adobe doesn't source pentesters for Reader on DICE. Microsoft doesn't source pentesters for SCHANNEL.DLL on DICE. Apple doesn't source pentesters for the iPhone bootloader on DICE. That's where the A-Team ends up.

What transparency are you adding here?

If the argument behind this was, "we're going to drive down the price of pentesting", that would be a coherent pitch, although I'd still want to hear how you expect this service will do that; again, the market is supply-constrained.

The transparency of who actually is good. The talent in companies ebbs and flows and the scores will reflect the work they are doing now, not what they did in their best or worst days.

There are many boutique companies that are excellent, but don't have a fair share at the market.

Companies that are 'all things to all men' tend to have quality issues over time... like the big security giants of the last decade. Eventually people get tired of it and look for specialists. That's where this will help.

Help me understand how what you're doing allows me to spot which Bishop Fox testers are "actually good"?

I'm not talking about "big security giants" like IBM and Deloitte. I'm talking about boutique application security firms that do little other than test software. They're already specialized.

So they get all the business they want from word of mouth and referrals? Good for them, but doesn't that imply that there's a lot of unmet demand?

Many big-ticket items are not transacted through a sales funnel beginning in google ads, even for individuals (businesses tend to have more formal processes). Houses, cars, etc. are largely purchased by going to a specific place (digital or otherwise) where such things are bought and sold. Not just googling "3 bedroom 2 bath house close to Redmond". I do not think this process involves a lot of unmet demand.

It's very much a supply-constrained market, which is why firms tend not to advertise much, and why it's hard to understand the premise behind this market.

Since this posting, I've had 3 boutique firms sign up. It is a supply constrained market, but a highly inefficient one.

I believe the market is inefficient, but what I don't understand is how something like this is likely to make it more efficient.

This is, for what it's worth, my field; I co-founded Matasano and helped run it until we sold to NCC.

Reply on news.ycombinator.com