Isn't this what "tiny stories" / Phi LLM are doing? https://arxiv.org/abs/2306.11644