Hacker News Reader

Comment by tptacek | Hacker News Reader

Where did you come up with that number? $500k is much more than a sitewide external app pentest of comparable scope would cost Apple, by an integer multiple. The bugs here are good, but they're not "bug bounty black swan" good; they're what you'd expect from a sitewide pentest.

I agree Apple got a great deal here (that's the point of bounties, and anyone who thinks they're a bad deal for strong researchers is... right). But I'm always going to point out that HN has weird misconceptions about the economics of this stuff.

View on HN

That second bug they describe would have allowed them to mess with inventory in a warehouse. They could have easily "disappeared" millions of dollars of products. Some of these other bugs would have required apple to disclose PII leak disclosure which could do tens of millions of dollars of damage to their company valuation.

You'll find, if you talk to people that do this work professionally, that bugs where you can tell yourself a story about the millions of dollars you could make are not uncommon, and that the rack rate for generating those bugs doesn't scale with their hypothetical value. I've done multiple projects for FIX gateways at exchanges. Those are fun stories to tell yourself! But those projects weren't even especially lucrative.

> where you can tell yourself a story about the millions of dollars you could make

It’s not about the dollars you could make. That’s probably pretty hard to get away with.

But the damage you can do? That’s a whole different thing.

Pen test that took 6 months with 10 people would cost at least $2mm using an extremely low $200/hr rate. People who are best in the industry will be significantly higher.

> $500k is much more than a sitewide external app pentest of comparable scope would cost Apple, by an integer multiple.

By a team of four experienced security researchers working for multiple months?

A classic false comparison: the four experienced security researchers working for multiple months covers 55 issues, not "that one issue".

If we're cherry picking a single one, the associated involvement and timeframe drops dramatically, to something much closer to one or two people, tops, over the course of just a few days, tops.

That's something a pentesting team can absolutely achieve for far less than $500,000 over the course of a few days, too.

I’m unsure what your point is? I see dozens of different issues listed in the post, on different endpoints, all of which presumably took time to find. When they said they had a team of multiple people work for months on this, I am unsure why you think they haven’t spent their time as efficiently as “a pentesting team”. Actually, I’ll be stronger: looking through the list of things they discovered, it seems like they were absolutely churning out vulnerabilities for the entire period. A real team would have certainly cost much more than what they’ve currently been paid.

A real team would have certainly cost much more than what they’ve currently been paid.

Yes, but that's a shared premise in this subthread already.

Issue count != time spent. I found about a dozen issues in a day once. And once, it took me three days to find one.

Always found at least a medium severity issue though.

Big engagements were typically a week, max. Usually one day of kickoff / getting “in the zone” for a project, three or so days of intensive testing, then the final day is usually writing reports (ugh, reports) all day.

Sounds about right. :-)

It's not. The median appsec engagement is ~4 person-weeks.

There's really 2 options here. One, Apple doesn't employ a pen-testing team currently, which would be nuts, or, two, the pen-testing team couldn't find these bugs, or they'd already be found.

Apple has product security teams, in infra security team that covers a lot of this web attack surface, a large red team, researchers, and employs 3rd party firms to do sitewide tests.

Apple is also huge, and no huge company avoids vulnerabilities; staff as ambitiously as you want, but any disjoint group of competent testers attacking a new target is going to find a disjoint set of bugs.

Or option 3: apple is HUGE, in all respects: physical space, people with access, code base, etc. etc. and they already have plenty of teams in place, but a bug bounty program is a cheap supplemental. In which case paying out more for your bug bounty program than you pay your real teams would be really weird.

Yes. I'd say "word to the wise", but I think very few people reading this thread buy pentest time in such large blocks: past a month and you start getting into steep discounts.

(This was not several months of full time work, but rather several months of part time work; but I'm stipulating the former condition.)

Your comment got me thinking, Apple probably was already buying large blocks of pentest time, and the comments in the thread make it seem like these were obvious flaws. Is that right? If we assume Apple already had a contracted pentest firm, can you speculate why didn't they find these flaws?

I don't know what "obvious flaws" means. I know from like a dozen years of consulting experience, and from 10 years of vuln research prior to that, that putting a different set of eyes on a target tends to get you a different set of bugs. Finding vulnerabilities is as much an art as a science, which makes sense when you think about what hunting for software vulnerabilities actually entails. If you could do it deterministically, you'd be saying something big about computer science.

I think we're on firmer ground saying that there are ways of delivering software that foreclose on "obvious bugs". But when we talk about fundamentally changing the way we deliver software --- in secure-by-default development environments, on secure-by-default deployment platforms, with security as a primary functional goal prioritized over time-to-market --- we're actually into real money now, not just another $250k on pentesters.

someone is watching schit creek

Yes, because it is worth in pentesting services 180k USD, no more no less. I mean, you can pay around 360k in London or SV rates and 180k in European for _similar_ skills people.

Calc based on 3 months, 5 people, 600USD/md rate.

EDIT as I can't reply to tpaceck below: no, those 2000usd/day rates do not exists in projects in size of 300MD like here. In general they do not exist for big projects.

Yes, I agree, you have rates around 1200 in high cost countries, yet as I wrote earlier, you can have similar/the same skill level at 600 usd/md if you're willing to work with guys not from HCC.

As to the skills I'm talking this level: https://research.securitum.com/mutation-xss-via-mathml-mutat...

If "md" means "billable day", a $600 billable day is extremely low for this kind of work; that's closer to what people pay for network pentesting. $1500-$2000 is closer to the market (before discount, assuming senior but not principal level delivery).

When I worked as a 'consultant' (glorified contractor) .Net developer, the company charged > 90 Euro / 105 USD per hour for my time. So that would make my going rate be > 800 USD / day. This is in a country where 50K / year is a decent developer salary.

I do not believe you can find pen testers worth their salt who would cost _less_ than a non-distinctive developer. At least not one who will do more than run some automated report over all your endpoints.

In that case, do you think that Apple is incompetent for not stumping up $250k or less for an external pentester to find these bugs? Plus maybe $100k more for an internal PM/point of contact for the pentester? Or do you think Apple handled it fine, the expected cost to the business of their security holes was less than $350k and they could just wait for them to come through the bug bounty program or for internal engineers to find them?

I think everything is complicated, and that is certainly isn't as simple as "Apple should pay paid $250k to a pentesting firm to find these bugs", because you could keep paying $250k over and over again and keep finding different bugs of comparable severity.

And finding these bugs of comparable severity isn't worth the $250k each time?

I can easily see the iCloud photo worming one making it's way into mainstream media and causing millions of dollars of reputational damage.

It's not a question of whether any spot assessment is worth $250k (though: Apple can get a sitewide pentest from experts for substantially less than that). It's a question of whether paying that continuously is worth it, or whether that many can be spent more productively on something else.

For what it's worth, "reputational damage" has always been a kind of rhetorical escape hatch from arguments that have become too mired in facts.

Reply on news.ycombinator.com