Clarkesworld AI Submissions Update | Hacker News Reader

Clarkesworld AI Submissions Update

neil-clarke.com

72 pointsem-bee34 comments

Seems reasonable, especially this part:

> The fact of the matter is that we have rules. Every publication has rules. Our rules say we don’t want generated content. They also say that we don’t want horror. If someone submits horror, you wouldn’t have a problem with me completely ignoring them, so what’s the problem here?

I hope they can remain open, but I have my doubts. In the end, there's money attached to it, and if people can take a shortcut on their way to make money - even when it's against the rules - they will. To use their analogy, if there's a chance people will make $1,000 by throwing away a fork, that fork's going in the trash.

Exactly, My SMTP server doesn't want spam per my policies, but spammers don't give two shits and I have to filter absolute massive tons of spam. I had one client where their domain was like that of a popular university. For every one valid message they received, they received thousands to fake addresses that didn't exist on the server.

> "As of a few minutes ago, we had received 576 submissions and processed 474 of them. 93 have resulted in bans and 7 have been marked as suspicious (a new category meaning they might have used those tools)."

I've had many stories rejected by Clarkesworld. but if I submitted something so bad they assumed GPT wrote it, I might die on the spot.

Unfortunately, I think this is a losing battle. They can't scale the number of human readers checking for story quality, and AI can scale almost limitlessly. Automated AI detection simply doesn't work. I've experimented with all the main AI detectors, and they are woefully inaccurate, especially with GPT-4 content.

>Automated AI detection simply doesn't work.

When did this change? As of 2 or 3 months ago, OpenAI's tool seemed pretty reliable at separating 100% ChatGPT content from 0%. Of course, past some level of a human using a generative AI and editing it and augmenting it, it's harder. And outside of certain artificial constraints in places like schools, it's not clear that using a generative AI assistant is even "wrong." Though to the degree that floods the web, publications, conference proposals, etc. with content that is just "OK" it makes the job of people filtering it who are already flooded with mediocre but not necessarily bad content that much harder.

All these publications constantly complain about being swamped with submissions, even pre-AI.

The top-tier ones should probably switch to mail-only for unknowns, with an allow-list of authors (those they've previously published, or whom they've rejected but are very interested in seeing more from) who may submit electronically. Hell, they probably should have before this.

Riffing on "mail-only", I'd further suggest:

- Include a printout of the story in the submission packet (shipping costs)

- Also require a signed (and perhaps notarized) affidavit stating that it's entirely the author's own work*

- Require a header/submission form to be filled out by hand (this can be verified quickly with a magnifying glass)

It's not perfect; the story could still be generated by AI, for example, and autopens exist. But it would put a very manual cost and effort barrier in front of their submission process, that IMO is still not so high as to discourage legitimate entrants.

* Obviously this should be informed by legal counsel, I am not a lawyer, etc.

As someone who has been on a lot of conference committees, I wouldn't necessarily require proposals to be sent physically; that would create a lot of work for us. But I do favor limiting the # per person and asking questions other than just the standard abstract that might discourage people who are just spamming everything in sight. Friction isn't always a bad thing.

I don't know that the extra work will be avoidable in the future. I think anything with a public API is going to be gamed and spammed.

Unfortunately, I think this is a losing battle.

This is just a narrow representation of what is to come. Everything positioned against AI will be a losing battle. It is a skill/creativity replicator and that concept breaks the existing model of everything.

The answer to this seems simple - require a micropayment along with the submittal. tune the amount until it's not worth spamming anymore. Return the payment when the submittal is accepted or rejected. Consider keeping the payment when the submittal is clearly AI or low quality spam.

We have all the necessary technology to do this today cleanly and efficiently. If we don't it's a social problem, not a technical one.

one of the important aspects for clarkesworld is international submissions. unfortunately , while technology for micropayments exist even in eg. many african countries, i haven't yet seen any micropayment system that works globally without high transaction costs.

I think openai needs to publish an ai Levenshtein distance api. Where you feed in a block of text and it tells you how close it is to something it’s ever generated. It can keep track of everything it’s ever generated and tell us if it seems familiar or not.

Really interesting, I appreciate Neil writing more about the process and philosophy than simply "we are banning AI tools."

As these models improve, I'm skeptical that we'll be able to differentiate human-written works from an edited work generated by a skilled LLM operator. It's already trivial to generate fiction content that scores as "100% human" on analysis tools.

However, sci-fi short stories are a particularly interesting case - by their very nature they are about investigating uniquely human experiences by using speculation to amplify things humans think, feel, and experience. A machine can never do that wholesale. But, what about a machine in tandem with a human that is articulating how they feel, and a premise?

Writing is supposed to communicate meaning; Spamming generated content with the appearance of meaning is poisoning the well.

I'm pleased that there is push back against this, but I don't think the people championing GPT are willing or even able to listen.

With fiction though the meaning that the reader gets often is not the meaning the writer intended, and often not even related to the meaning the writer intended. The story often serves more as a prompt to get the reader to create meaning than as a way for the writer's meaning to be communicated to the reader.

It's not clear that real meaning from the writer as opposed to the appearance of meaning matters as far as how well the story works to get the reader to generate their own meaning.

100% relevant: "Someday" <https://en.wikipedia.org/wiki/Someday_(short_story)> by Isaac Asimov, 1956

I think a AI-coautored section would be interesting (with an additional field to let people describe their writing process, from "I just asked for the greatest SF short story ever" to "I gave it my draft and used it as an editor").

While a lot of this writing is spam trying to make easy money, I do think that some people are doing cyborg writing in good faith and would happily report their writing as such.

This is similar to the older discussion about using Photoshop in photography competitions. I think that eventually moved to "everyone photoshops somewhat", which is likely where the AI tools for writing will go.

I mean, how many people use Grammarly? Doesn't that suggest changes to test based on (somewhat) ai?

I have started giving GPT some of my own sentences I am unhappy with in prose and asking for it to repeat it in a more "active voice". I do not paste it into my writing -- I read the results and edit my original sentences as I wish, if I wish. This process will likely find me reaching out to GPT less and less as I become more familiar with the writing style I am seeking.

I think methods similar to this will become more popular with writers.

Understandable but would be interesting to have a "State of AI" writing post every once in a while to see how AI measures up. Once AI starts making competent short stories, I can imagine them being good enough to write quests for next mass effect etc. So far AI content been pretty generic on that front.

The only answer to this is to crowdsource curation. A small editorial team isn't going to be able to do anything other than cherry pick from a trusted pool.

Crowdsourcing curation leads to lower standards and misaligned incentives; it's trying to fight the problem with itself.

the problem with crowdsourcing is that it would probably be a pre-publication which would make it difficult to submit the story elsewhere because many magazines (including clarkesworld) do not allow that.

currently they have multiple slush readers, and depending on volume they could engage more as long as the volume does not grow indefinitely.

btw, here is a thread that gives some insights how slush reading works: https://www.fantasy-writers.org/forum/slushpile-answers-clar...

Crowdsourcing doesn't work if you want to produce a Clarkesworld issue, because Clarkesworld has an audience expecting stories of a certain style and sense that requires training for readers.

Wouldn't this be (at peast partly) resolved when it comes time to sign legal documents? At some point, the submitter has to legally vouch for their identity and authorship.

You could front-load this requirement, of course: provide proof of identity (tech already exists), then legally vouch for authorship at time of submission.

TL;DR: Raise the barrier to entry, instead of scaling the ability to review.

A contract without teeth is worthless.

Who in the world is going to sue someone for submitting an AI-generated story to a sci-fi magazine? Who has the time, resources, and motive? You may as well just ask everyone to pinky promise that they didn’t use AI.

I agree, but I'm suggesting more than a contract; I'm also suggesting identity validation. Just the act of requiring legal proof of identity might weed out most of these attempts.

Perhaps most authors wouldn't care, but identity validation would gut the concept of the pen name. And as beneficial as pen names are (allowing early women authors to publish books at all, providing an author an out if their name becomes too associated with one genre, protecting authors from fanatics), eliminating them in an attempt to prove whether a tool was used in a books creation is less than ideal.

And worse, I have my doubts that it would even work. See: counterfeiters on Amazon.

identity validation would gut the concept of the pen name

as far as i understand clarkesworld submission rules, you may submit and be published under a pen name, but have to reveal your real identity to the magazine, which they will keep to themselves

That’s not how pen names work in publishing. Source: I’ve published books both under pen name and legal name.

Reply on news.ycombinator.com