"AI use detection" is, like any test, not without cost. Meaning that, as a teacher, accusing a student of using an LLM, it may be prudent to consider the cost of a "false positive" accusation. I've seen a couple of examples now where students find sudden spurts of motivation and show unexpected talent on an assignment, to be accused of AI use after handing it in.

One should ask oneself: How many insults to the intelligence and creativity of an unexpectedly excelling student (that hasn't used AI) is it worth catching the shortcut-taking, LLM-using student? Is it 1/10? 1/1000? How much "demotivation of an unexpectedly excelling student" is the "rightful punishment of the cheating LLM using student" worth? And what is the exact cost of a false negative (letting the LLM using student off the hook)?

In other words, where on the Receiver Operating Characteristic (ROC) curve do you want to sit, as a teacher? I imagine it's quite the dilemma.