00—1A: have feature flags A and B
1B-55: have feature flag A only
56-FF: have no feature flag
So the actual gotcha here is that there is no cohort for "feature flag B only", right?
This setup can actually be desirable if feature B depends on feature A.
And that, as you add more tests, user 00 will always get the test treatment for every test. If you're running a lot of tests which introduce experimental features or changes to workflows, user 00 is probably going to find the site a lot more chaotic and hard to understand than user FF, and that will skew your test results.
Yep, exactly - by "that first set" I meant the 00-1A group, could have been clearer. Whatever the smallest rollout bucket is, that group is guaranteed to have every single feature.
This was quite a while ago, but I think the actual case we noticed this with was several features released to 50% of the userbase - so every single user either had all or none at once (unintentionally)