At Microsoft back in the day, we called those “bug bashes”, and my startup inherited the idea. We encouraged the whole company to take an afternoon off to participate, and gave out awards for highest impact bug, most interesting bug, etc.
Is the secret that it only works if the entire company does it, like you suggest?
And yes, I completely realize that Scrum is terrible. I'm just trying to work within a system.
If you're competing within the team to complete as many story points as possible that's pretty weird. Is someone using story points as a metric of anything other than forecasting?
Very nearly every company I've worked at that uses Scrum uses story points, velocity, etc., as a means of measuring how good you or your team are. Forecasting is a secondary purpose.
I picture a construction company counting the total inches / centimeters each employee measured every day. Then at the end of the year firing the bottom 20% of employees measured in total units measured in the last 12 months.
I've seen that justification time and again, and it feels disingenuous every time it's said. (Feels like a corrolary to No True Scotsman.)
I've also seen scrum used regularly, and everywhere I've seen it has been broken in some fashion. Enough anecdata tells me that indeed Scrum, as stated, is inherently broken.
Solution: don't measure story points
But assuming that everyone cares about quality (I know, a big leap), what has worked for me is: tagging stories as bugs/regressions/customer-found-this and reporting on time spent. If you're spending too much time fixing bugs, then you need to do something about it. New bugs in newly written code are faster to fix, so you should be able to show that bug bashes make that number going down quarter over quarter which contributes to velocity going up.
Alternately (and not scrum specific) I've had success connecting a CSM/support liaison to every team. Doesn't give you a full bug bash, but even one outside person click testing for 20m here and there gets you much of the benefit (and their incentives align more closely with QA).
It wasn't perfect, but it worked well.
Granted, it required a very specific management, devs with the right mindset and constraints on the kind of projects that could be done (anything customer facing with a tight deadline was off for instance. We used that for the internal infra). So I don't see how you would build a plane at Boing with scrum for instance. Or anything that require very tight coupling af many teams (or hardware).
But for us (60 devs in a company of 200), Saas, it worked great.
Not only do we uncover bugs, it’s a great way to get the whole company learning about the new things coming and for the product team to get unfiltered feed back.