With my partner we have been working to invert the overall model.
She started grading conversation than the students have with LLMs.
From the question that the students ask, it is obvious who knows the material and who is struggling.
We do have a custom setup, so that she creates an homework. There is a custom prompt to avoid the LLM answering the homework question. But thats pretty much it.
The results seems promising, with students spending 30m or so going back and forth with the LLMs.
If any educator wants to Ty or is interested in more information, let me know and we can see how we collaborate.