This is a brilliant idea. One of those "why hasn't that happened yet??" ones.

Is there any way for me to obscure PII data easily? or anonymise any data recognised as file contents or base64? That'd help.

Have you considered hooking up alerts. For example to run `npx pythagora --script ./path/to/your/server.js --mode test` but with an option to pass a key/function to send alerts(perhaps just picking up test fail and dealing with it in the CI is better though) if a test fails that previously passed (wherever that command is run, so in the CI or if manually ran).

We do use socket.io for a massive amount of our requests so I don't think it'd cover that (happy if wrong?) but I will look into if we can add a pull request for socket.io support, if you are interested in that scope creep... As that would be very powerful.