Hacker News Reader

The most absolute simple form of production testing is simply trying to use the application to make sure all the parts seem to work based on what you'd expect from a glance. This seems quite distinct from randomly terminating a service or injecting random bytes to see what happens.

I think there's a gap between running tests without "defined starting and ending states" and chaos engineering. This would be closer to exploratory testing for me, however rather than having a QA team run them you're outsourcing to the users at the expense of risk. Everyone does this (it's called a release) and you can deal with this in two ways - have tracing and monitoring in place to collect the logs from your unpaid testers or ignore it. Add techniques like blue green or canary deployments and you somewhat reduce the risk cost. However this cannot be a replacement for decent prior software testing practices.

In a multi-tenant SaaS application, it should be possible to have a predictable begin and end state for a test case. Once you get this, then it's not to hard to make some canary tests periodically check the most important corners of the system.