Periodically pull up some randomly selected documents and ensure that they match the corresponding records. If you can’t find the documents or if they don’t match the records, you trigger an audit.
The aphorism we used was “nobody needs a backup system, but everybody needs a restore system.”
We have self-service backup restores for user files, so we get to spot check which users can be bothered to read the documentation on the wiki...
So, for example, items that people tend to think would be incorrectly responded to by people with dementia, but are actually easy for people with dementia to respond to. Or items that sound like they'd be endorsed by people with severe mental illness but are not.
In more sophisticated implementations, it's not just responses to these types of items, but multivariate patterns of responses that are unusual that are flagged. So, for example, not getting items increasingly incorrect with increasing difficulty (i.e., getting easy items incorrect and hard items correct) might be flagged, just for one example.
Specific examples are hard to list because (a) it would violate test copyright and/or other ethical issues, and (b) are sort of idiosyncratic and require actual data on the items.
With cognitive testing for example, lots of times certain types of memory are intact in severe brain injury but people who are faking don't realize it.
The the items in an effort/response validity measure reflect some kind of systematic schema on the part of respondents. It's very non-arbitrary.
The brown M&Ms in contrast are sort of trivial and arbitrary. That is, there's nothing about M&Ms that has anything to do with actual stage safety or management, and the assumption that attention to M&Ms and stage safety are the same is kind of odd.
In fact, one might argue that the brown M&M clause pulls for a sort of cargo cult phenomenon, where people are following something to the letter without regard to deeper principles or understanding of what's important or not.
Sometimes psychological tests do have items that test for random responding, which is maybe more comparable. But even then I don't think it's the same, because it's still item responses that are critical. To me the brown M&M test is like looking at the response time to an initial consent page, and arguing that if that is unusually long the person probably isn't responding to the items per se.
I think there's maybe something to the brown M&M test but to me it pulls for the wrong things. Why not just actually, you know, make sure they abide by the long list of supposedly safety-critical items that you specified? If you as the performer are so sure that they're safety critical, but can't be bothered to check those, and instead rely on brown M&Ms, because it's easy to check in your dressing room, who's the one being lax about the stage?