A few others have mentioned CUPED (Kohavi et al. 2013), a way of doing exactly what this blog post says you can’t. CUPED is a great starting point for anyone new to the subject of pre-experiment control variates.
The post is bold enough to describe any use of control variates is a “fallacy” but somehow doesn’t mention the most famous method for doing this (CUPED). The author is only interested in linking to their own prior blog posts, and does not engage with any reasonable (or attributed) argument for the method they’re dismissing.
The post’s main argument is that using CUPED-style control variates provides no benefit when N is sufficiently large, so we should just make N bigger — as though “sufficient N” grew on trees.
The next argument is less silly, but still wrong: They note observed differences in pre-treatment effects are “random fluctuations”, and claim they thus cannot be used for anything. Their argument ignores the reason pre-treatment measurements are subtracted in the first place: In CUPED, we acknowledge pre-treatment observations are undesirable noise, but presume this noise already contaminates our post-treatment measurements. We collect pre-treatment measurements exactly because we want them gone — the pre-treatment measurements tell us how much meaningless noise to subtract.
As a reductio ad absurdum, imagine an “A/B test” (an RCT) of a drug that ostensibly makes humans taller. Would it seem reasonable to you, as a participant, if the doctors never ask what your initial height is, and only measure you once the trial is done?