* the core idea is to selectively use training instances to learn a compact versions of a model. Multiple models are produced with different tradeoffs between accuracy and size. The user is then free to pick one that fits his needs. See [2], [3].
* one can also find the "minimal dataset" a models needs to reach a certain accuracy - [4].
Currently working on:
* I am reformulating the problem to work with (twice) differentiable losses to speed-up the compaction process.
* Also, I am looking multiple "stages" of compaction to see if that leads to models with further reduced sizes.
Refs:
[1] https://compactem.readthedocs.io/en/latest/
[2] see how the model's generalization is affected, also has accuracy vs size curves: https://compactem.readthedocs.io/en/latest/key_ideas.html#wh...
[3] Compaction profile - a plot to visualize the range of size reductions between the original and compacted models: https://compactem.readthedocs.io/en/latest/usage.html#compac...
[4] https://compactem.readthedocs.io/en/latest/usage.html#optima...