Hacker News Reader

Hmmm but if you change the distribution, this won't be deterministic anymore unless the randomness only applies to new users entering the test setup.

Which is necessary. You don’t want to compare two wildly different users. This is why you need to define criteria for eligibility.