Ask HN: Is there any work on self-testing software?

For clarification, I don't mean automated testing, test suites, or TDD. I mean more like HAL: "Dave, I have detected an error in one of my circuits." I.e., software that, when run in normal production mode, updates its test fixtures with the latest environmental changes, and then runs its test suite on itself. If it passes, then the full production mode continues.

This is an idea I've had which would be a really nice advance for my field: parsing and scraping public government legal texts. I'm wondering what others have done like this.

Unfortunately, when I search for "self-testing software" I get only _automated testing_ ... even Martin Fowler using self-testing as a synonym for it: https://martinfowler.com/bliki/SelfTestingCode.html.

Thanks!

----

Details for the curious:

I have the problem of false positives when (say) the Nevada legislature changes the format of a statute. My program completes without error, but the output isn't correct. (My take-away: I've allowed an illegal state to become representable, and my code needs to change.)

But I noticed something interesting: when I update my text fixtures (they're full copies of a few of the Nevada HTML input files), I get test failures! This is great --- no false positives here. And it's nice confirmation for me that I've got a good test suite. (I did write this code using TDD and was careful not to code ahead of my tests.)

And so I had a thought: I could automate this. When I run the app ( https://github.com/public-law/nevada-revised-statutes-parser ) for real, it can update these fixture files and re-run the test suite to see if it still holds up against any recent changes. This wouldn't be a 100% guarantee of correctness because the source texts may have changed in some way that happens to avoid my tests. But it definitely would have saved me in this situation --- instead of finding the error way down the line with the data already in use.