For example, LLMs love to talk about LLMs (and the people who write with LLMs love to write about LLMs). Could "large language model" itself therefore be flagged as an AI-like phrase by this approach? It didn't exist much in the literature before 2022, does now, and certainly does more in AI-generated text: but, it is not actually a great way to distinguish modern AI generated text from human written text.
A helpful control would be to show that on some cohort of papers that can be declared reasonably clean of LLM generated text post 2023 there are very low rates compared to the arxiv.
For example, while papers in the journals Nature and Science are unlikely to be entirely LLM free at this point, if those were tested through 2026, we should see a line significantly lower than the arxiv's growth.