Hacker News Reader

Any chance you could point me to a reference? I'm doing research in this space and currently working on a paper which does exactly this for diagnostics of testing processes.

View on HN

I'm not sure which thing I mentioned you need a reference on. For p-values on machine learning techniques, http://vovk.net/cp/index.html I'll eventually do a blog post on this subject; it's very good math that all ML people should know about, though Vovk, Schaefer and Gammerman write pretty dense articles.

For statistical power... "Statistical Power Analysis for the Behavioral Sciences" by Jacob Cohen.

Sorry, I intended to quote your last sentence, applying a confidence interval to a reinforcement learning system, especially with respect to multi-armed bandits / adaptive experiments, but if I have to dig in to some signal processing stuff I am happy to do that.

It seems the conformal prediction link has some relation. I will dig, thanks.

Yes, conformal prediction does exactly this.

Reply on news.ycombinator.com