Software changes before my eyes constantly via normal releases so I don’t really care if I’m part of an experiment or not.

That’s software though—I see your point for something like content. I’m already used to seeing the title or thumbnail of a YouTube video change as the result of an experiment “winning” but the content itself…that would very jarring.