I'm sorry that's been your experience. What should have happened is a primarily manual penetration test, administered by security engineers who themselves used to be fully competent software developers. Any automated tooling should have had strict manual verification and should not have been the focus of the test. Furthermore, superfluous results should not have been submitted in the PDF report.

Strictly "policy" audits such as PCI compliance differ a bit, but in general they should still involve a technical deep dive into your product's infrastructure, conducted by consultants with expertise in multiple tech stacks and overall experience in a variety of frontend and backend frameworks.

The final deliverable ("PDF report") also should have been hand-written, and in language that conveys technical expertise, complete with recommended steps towards remediation of any issues.

My employer, Accuvant does this, as well as Matasano (more well known here on HN).

As for why it's so expensive...well, I bill out at about $2000 per day. It really comes down to a lot of what people like patio11 and tptacek like to talk about here regarding consulting:

1. This is highly specialized work, with a much smaller population of competent engineers than typical web developers (for example). As such, it naturally receives a higher fee for supply and demand. Now, some people abuse this and run scans like Nessus and call it a day. These are not real infosec firms, they are parasites.

2. More specifically, we ask for it and we receive it, and we do exceedingly well. If people keep paying us five figures a week to perform a penetration test, we're not going to stop asking for it or reduce our prices.