I think that’s a bit extreme. What if I’m doing analytics via server-side logs rather than client-side (e.g. Google Analytics)? Are we supposed to ban server logs too? What happens when I want to actually troubleshoot issues with my site?
A/B testing / etc is a form of human subject research, the only way to experiment on humans ethically is to have voluntary informed consent. The fact that you're 'merely' running psychological tests on website users instead of performing medical vivisections doesn't free you from the moral and ethical obligation to acquire voluntary informed consent.
Tell your legal team you want the opt-in legal language to be easily readable and understandable by the average person and make the font big so nobody can say there was fine print.
Advantages:
- People that opt-in to such things are far less likely to make a stink when something breaks. Again, reward them for finding bugs even if its something trivial. Maybe even set up a private community forum for these people to discuss the bugs they find.
- These people will also go out of their way to find bugs for you. Real-world traffic meets a free Quality Assurance team.
- Your Beta/RC URL can be excluded from any service level agreements in an updated AUP/ToS. Finance people like such risk mitigations.
Disadvantages:
- More work and cost up front to set this all up even if it pays for itself in the long run.
- Requires a bit of humility and discipline to openly share your errors with a handful of your user base.