Humazon
hughhowey.com
hughhowey.com
Storyscope (https://github.com/jenna-russell/storyscope) is a really interesting concept, and one I'm in the process of reproducing and expanding on, even though it is less accurate on direct detection than something like BERT or Binoculars. It is interesting because the usual obfuscation tactics, even having a human transcribe and rephrase the story don't work; it looks at the story itself, rather than word use and grammatical quirks, etc. Plot, agents, temporal structure.
So far, AI writing is detectable to a pretty high degree, though I think it'll be a war of attrition that AI eventually wins. And, all the detectors are the same tools one could use to create undetectable AI writing; AI loves to iterate in a loop, seems like iterating to rewrite to be undetectable is a soluble problem (though models currently probably would end up writing worse and worse to avoid detection, and the current best models have begun integrating watermarking, pushing back the defeat of human writers for some time).
I'd like to come up with a more fun game loop, but nothing has revealed itself to me yet. Shorter passages are easier to gamify, but shorter passages are much harder to detect.
> Because to me, authenticity makes for a better reading experience.
This pair of sentences gave me pause, until I realised that they're not in contradiction - the author is deliberately non-judgemental of _others_ making a personal choice that _he himself_ would prefer not to make. Remarkably rare, nowadays.
But he didn't. And it didn't affect his experience until he found out.
Which strongly suggests there was nothing wrong with the reading experience, and no lack of "authenticity" in the text itself, on its own terms.
It was a chimera.
As far as most readers knew, Carolyn Keene, Franklin W. Dixon, and Victor Appleton were all authentically real people.
Discovering they weren't didn't change the quality of the writing - it only changed the implied relationship with the author.
But that's a parasocial experience, not a feature of the words on the page.
I suppose the implication is that because the book went through the mill the author and publisher somehow didn't really mean them.
Perhaps they were tainted by money instead of being products of pure enlightened generous impulse.
But how can you not mean something like a Tom Swift book?
The series was incredibly successful and had real social influence. Would it have been more successful if Victor Appleton wasn't a made-up person?
Why do we expect authors to be scrupulously physical, when creating imaginary people and imaginary worlds is literally the job description?
Especially as I feel that more and more this degree of transparency will be crucial for people to make more informed decisions in what they wish to read - and by the author's point, where do they want to spend their money.
What if an individual not knowing any better could absolutely enjoy AI written stories from cradle to grave, but would rather live in the blissful innocence of contributing to a lone writers' income rather than content farms? Isn't that enough reason to want transparency here?
My point being, if there's really "no harm" in the deception of either human-powered or AI-powered content farms. Then let's opt for imposing the transparency for readers to truly have the choice.
> But that's a parasocial experience, not a feature of the words on the page.
So? Consumption of media does not exist in a vacuum - how one feels about the work is inextricably affected by the context in which is was encountered and how one views it. We feel prouder (subconciously or not) of reading works that are considered "great" or "weighty". We feel nostalgia for rereading books from our youth, or affection for a book lent by a loved one.
The writer is not saying that the words _themselves_ were changed by the new information, but that his perception and experience of them was. That experience _is still real_.
> Why do we expect authors to be scrupulously physical
"We" don't. Some people prefer it - others don't care. You're free to fall into the latter camp if you want; but the former is a valid preference, and one which _could_ be served by a market.
- blockchain attestation [ crypto-graphically signed by human author / artist ]
- proof of work in the traditional sense, eg video of artist at work, stages of the painting from sketch to under-painting, brushstrokes, layers, glazing
In my case, I think videos of the painting process are a reasonable, if temporary, solution.
I agree with Howey's sentiments. AI-written books aren't bad in themselves -- they're not ontologically bad -- but they tend to be, by their very nature, derivative. Narrative quality is, as yet, still quite poor. And self-publishing on Amazon.com is full of that stuff. (I've long since stopped buying self-published stuff on Amazon on account of low-quality content, and Amazon's review system seems severely broken.)
Humazon would be a welcome thing. Or even a "human written" flag for books on Amazon.
--------------
To do what the author of this post asks for is incredibly difficult. You can't reliably use a LLM to detect the output of other LLM's. You can't trust user reviews to distinguish AI from human writers because AI can easily produce reviews. You'd basically need to form a central gate-keeping authority that decides who is allowed and who isn't. This would be resource intensive and fraught with problems. (e.g. How many authors would sue if they were blocked as the result of a false positive? How many using LLM's to write their material would brazenly sue for being blocked too?)
It really bothers me that one of the biggest problems with LLM's is how difficult it is to filter out what they produce.
Turtles all the way down I swear.