The only reason to care is that the implicit proof-of-work signal has broken because LLM text is so cheap. Open forums might need to be pay-per-submission someday...
It feels like we're happy to take the first as a surrogate for the second, or at least being good at the first drops our guard on questioning the second.
And it should be illegal to return machine-generate text in response to a discovery request.
Request: "Disclose all documents written between apr 1 and apr 12 regarding topics x,y,z."
Response: "There are 12 billion documents matching those parameters. Here they are."
Request: "Please respond with all documents related to X,Y,Z".
Response: "There are none."
That being said, automatic detection seems like a lost cause.
Also we can use more than just the text output. A human writer doesn't generate a piece of text in one pass. Instead they go through the drafting and editing process. We can design devices to capture keypress or pen stroke(iirc ther were studies on fraud detection based on keypress patterns/mouse movment). One can attempt to train a new model to mimic themselves so we need to somehow make sure that the amount of training data required is too much to be worth the effort.
For downstream testing, the goal isn't mainly to verify whether a piece of text is AI genereted but to make sure that a student who can pass the test would essentially have to know the material sufficient well(so this defeats the purpose of cheating).
We don't need ML-generated-text detectors. We need BS detectors. If they have false positives and trigger on human-generated BS, that's just a great side-benefit.
"This comment could have been written by an AI" is good enough reason to exclude it. That which does not need to be said, need not be said.
As that XKCD's title text says: "And what about all the people who won't be able to join the community because they're terrible at making helpful and constructive co-- ... oh."