The documentation is still very much in progress, but I thought I'd share this
library/DSL that I've been working on, which is inspired by my experiences in academia
and quant finance, as well as a desire to improve on scikit-learn's "pipeline" module.
I'm hopeful that others might see the benefits of this style of writing data-modeling
pipelines, or perhaps critique it mercilessly :)
One point of inspiration here is my experience that many researchers and data
scientists in industry, while fastidious about the in-sample/out-of-sample distinction
for the predictive "core" of their model (i.e. the regression or the ANN or whatever),
are often less conscious of the same distinction for all of the feature-preparation
steps and prediction transformations that may precede or follow that core in their
over-all pipeline. For example, before fitting your regression or whatever, you might
z-score some of the predictors, and when you later apply that now-fit regression to make
predictions on some held-out data, you really ought to z-score the held-out feature
values using the same means and standard deviations that were "learned" at fit-time
(they should be considered part of the "state" of your model). But this is often
inconvenient or overlooked by practitioners, who might naively z-score the held-out
batch using its own means and SDs, before feeding it to their trained model.
But if you imagine receiving the held-out data not as a batch, but as a stream of
observation one at a time, and needing to generate a prediction for each as it arrives,
it's clear that this is not correct! (As you make each prediction, you can't know what
the held-out batch's means and SDs "will be".)
So with Frankenfit, I wanted to make it easy to write end-to-end data modeling
pipelines where this distinction between fit-time and apply-time is baked in all the
way through the intermediate transformations, be they z-scores, winsorizations,
imputations, or whatever. And it turns out that this can also make for very elegant
expressions of common resampling workflows like cross-validation, hyperparameter search,
sequential learning, and so on.