There are systems and processes to help with all of this of course, but at the end of the day it's still a pretty tricky job to get perfect all the time.
There are systems and processes to help with all of this of course, but at the end of the day it's still a pretty tricky job to get perfect all the time.
This comes from a variety of experiences: I used to manage a bug bounty for a mid-size company on Bugcrowd; in 2014 I surveyed people managing a bunch of programs across different sizes; I've participated in bug bounty programs for companies of different sizes.
The more you offer for rewards and the more recognizable your company name, the more you will be spammed by people submitting reports like (I kid you not): "You have the OPTIONS method allowed on your site this is really serious." The last time I looked at the numbers, Google had over 80,000 bug bounty reports per year, with about 10% of them being valid and maybe another order of magnitude being high severity (I'm fuzzy on the last bit). It's probably over 100,000 per year at this point. It's not uncommon for recognizable but smaller companies to receive one or more per day.
I'm aware of full-time security engineers at Facebook and Google who do almost nothing but respond to bug bounty reports. It's a lot like resumes - people who have essentially no qualifications, experience or (most importantly) a real vulnerability finding will nevertheless spam boilerplate bug reports to as many companies as they can. Take a look at the list of exclusions on a given program - you'll see that many of them explicitly call out common invalid findings that are so ridiculous it's kafkaesque.
HackerOne and Bugcrowd provide a lot of technical sophistication to prime companies for success, but there is an organizational component that is very difficult. If your program is very active, it requires dedication to tune it so you're not flushing engineer-hours away responding to nonsense. This is not to say they're bad - quite the opposite, I think they're fantastic. But I generally recommend smaller companies set up a vulnerability disclosure program through a solid third party, and do so without a monetary reward until they can commit to dealing with a reasonable deluge of reports.
One other thing that never really gets any press is the fact that a good chunk of the folks sending in reports are young people in impoverished nations. Some of them can be pretty tricky to deal with, but if you hold a hard line on professional expectations you can see them flourish in pretty short order to be some of the best reporters out there.
I only spent a short amount of time on the program I was with, but it was very rewarding. A+++, highly recommended.
Even at this low rate, it is not too bad. Let's say you receive 10 reports. You can relatively quickly identify the 8-9 noisy reports to find the 1-2 valid ones. Of course, a higher SNR is always better. It saves you time and effort.
On HackerOne, the average SNR across all programs is over 30%. The platform can automatically filter out certain reports that are duplicates or out of scope.
The platform maintains an average signal rating for each hacker (aka security researcher). Companies can limit access to their programs to hackers with a certain signal or higher. This will significantly increase SNR for the program.
Companies can also opt for a HackerOne program with triage included, in which case the SNR rises close to 100%.
So if a new user of the platform, finds a valid or high impact bug, will be unable to report... less noise but a high value bug unreported in that case...
I'd perhaps accept 'I was so deep in the code!' once from a very junior developer, as a learning experience.
To the person that reported a bug, that one report makes or breaks their entire opinion of your organisation. We lost customers because of poor communication, and on the other hand made some very happy repeat-customers even when we had to say 'we can't fix that yet' - but they were in the loop for the whole process and understood why.