I guess different small models will have different points/goals, but you can still have a small model with lots of training effort or a large model with little training effort.

I think the point of most (frontier) small models is usually to provide the best answer possible given small inference resources, rather than to reduce training time.

This is more of a toy model, so fun and an interesting project but it doesn't necessarily tell us what the art of the possible is for small models.

For one thing, the model is trained on a language modelling task, not a question-answering task?

As I understand it, the most effective small models are synthesized from larger models.