Hacker News Reader

Hacker News

Comment by sieve | Hacker News Reader

Parent Full thread

sieve

> inconsistency across various parts of the story, items that are introduced, questions that go unanswered etc.

This is a workflow problem.

What the LLM normally needs:

- very basic outline: what the story is about

- basic cast of characters: like fiction/plays of the 18-1900s

Then you go chapter-by-chapter. But the most critical thing is that you MUST provide the chapter beats. From start to end. Location, environment, events, any critical conversations. For a chapter that is 2-3K words, this chapter outline often becomes 2-400 words or more. Because you have the story in your head, you can handle the stuff that the LLM is likely to screw up.

There are ways to build a system where the LLM can keep track of many things, but then that becomes programming, which is just as boring.

View on HN

I had an amazing AI system that did this and worked well for me.

What you did was you started off with the high level vision of the story, you'd figure out the characters in detail, supply their motivation, think about how those things might conflict with each other etc. Then you'd think about how to add various plot points into it so different characters with motivation might find productive tension with each other. You map it out chapter by chapter, what beats go where, what things happen to push interesting conflict.

But the amazing thing about this AI system was it let you go deeper, you could go into individual conversations and actually place the words people said to each other, add color to the words by appending it with stuff like "she said, hesitatingly". Entire descriptions of scenes could be vibe generated by you imagining in your head exactly what it would look like and finding perfect words such that the image in your head would be transmitted over to the reader.

Finally, the AI system had a way to inspect the psychological state of people who might want to read your work by recruiting people who are friends of yours to read each work in progress and you can interrogate them just like a chatbot over what worked well, what didn't, did you understand what was going on over here? Did you feel invested in the characters? You can use this feedback to spin up a brand new version.

Anyway, all this stuff is a lot of work but don't believe the people who say you can just press a button and get a good story out. If you put in the effort, this AI program called notepad.exe can have you generating genuinely human sounding prose!

I won't go into too much detail, but I have a plot outline, development bible, story mountain, character outlines, style guide and more. I have four agents running on various tasks; writing, error checking, consistency and critique. They can generate a good story but there are holes and issues so it's far from perfect and I wouldn't give it away for free, never mind put a price on it.

To add to this, in my 9-5 I work as a designer. Colleagues in other parts of the business sometimes come to us with stuff they have created using AI tools. To a layman they look really good, but if you know what to look for you can spot the issues.

At this point in time my opinion is that any AI generated thing is great at making fools of the unaware, or those who are not looking at something critically. It can create something that looks like a duck, walks like a duck, quacks like a duck, but on closer inspection it's not a duck.

I tried the programmer's method. Automating stuff. Then I realized I prefer the wood carver's method. So I take it slow.

I think and write the outline for a chapter. Then I have the LLM produce it. I read the draft. Tweak things. Reroll. Take the best from multiple rolls and merge them to produce a organic whole. Manually patch the minor differences in pacing and tone.

Rinse and repeat.

It takes 100-250 hours across 3-4 months to produce a story. But I drop 99.9% of the stuff within the first day or two because the plot or characters do not work.

I use the web front end directly. Let's me switch between any model including Gemma 4 running locally.

If you have all that already why don't you just write the book yourself?

I think that's probably the best approach right now.

No, this is a problem with the nature of LLMs and the text they generate.

I do not expect the LLMs to do everything. They are assistants. You still bear the responsibility of stitching everything together into a cohesive whole. Make changes as required.

That is where expectations differ I guess.

But they generate objectively bad text. Why do you need an assistant to do that?

Perhaps for some uses (say a summary of some business document, or generating code which is then discarded or edited) that's acceptable. For literature, do you really feel they are useful tools? So many changes would be required IMO that you're better off just writing the story you want to tell.

> But they generate objectively bad text.

By default, yes. That's exactly the same for code: by default, even with planning and patient nudging towards best practices, you get passable code at best. Not elegant, not performant, and not particularly readable, either. Basically, an uninspired salaryman type of code.

In both domains, you can get much better results in some specific circumstances. Prompt, skills, memory, and the task must align, but when they do, you can get good quality building blocks that you can then work with. It's crucial to recognize the instances where there's a chance of getting better-than-average results and ones where you could dump your whole week of tokens into and still end up with a mess. This is a skill of the LLM operator.

Once get that skill, the only issue is capitalizing on it: basically, how you fit handling of the generated building blocks into your workflow. If it's seamless, it can be a big win. If it's not - yeah, writing by hand from scratch is often better.

I’ve certainly heard from LLM operators that there is a lot of skill in developing prompts and guiding their agents, but frankly it doesn’t seem like there is much benefit at that point unless you’re willing to accept mediocre results or spend lots of time rewriting and trying to find errors.

Given the examples of writing I’ve seen so far from them I’m skeptical that LLMs are useful for any writing where quality and truth are important.

> doesn’t seem like there is much benefit at that point

That depends on the problem you're trying to solve.

I found it to be of great benefit when trying to overcome "writer block". Mediocre prose - but at least some prose to work with, tailored to what you want to write - can be a much better starting point than a blank sheet of paper.

Also, editing can be fast: the output is mediocre in predictable ways; it's not hard to quickly identify everything of value in a paragraph and rewrite the rest. It probably takes longer than writing the paragraph from scratch - but only if you already know precisely what you want to write, which is far from "always" when writing prose, IME.

> But they generate objectively bad text.

They don't. I don't think people are taking full advantage of their capabilities. They produce perfectly functional prose. Sometimes, they produce something genuinely amazing to read. But that is quite rare.

People should stop expecting brilliant prose from LLMs without doing the preparatory work themselves first. Very few human authors can manage that in any case

I’d partially agree. They can generate very good descriptive text however the issue I’ve found is that when something is implied it’s not fully ‘understood’ in the context of everything else and then gets ignored by the agent.

My solution to this, as the human-in-the-loop, is pretty simple. If the generated draft is missing something that was implicit in the chapter outline, I ask the LLM to incorporate it and rewrite the para/section. Then I fix the outline to make the point explicit so that a reroll does not make the same mistake again

Show me some examples of good long form text please. Your hedging in reply (perfectly functional) does not inspire confidence.

This is something from early 2025. The situation and my own way of driving the models has improved somewhat in the intervening period.

Variations on a Theme of Saki

https://gist.github.com/s-i-e-v-e/b4d696bfb08488aeb893cce3a4...

I’m sorry but this isn’t very good, problems from a cursory look:

Stuffed full of adjectives, a common problem when LLMs attempt to make the text literary.

Stuffed full of incidental detail (window inexplicably open), which doesn’t quite make sense - why would you leave a window open for people to return?

Things which suddenly loom large in the narrative without any previous mention (roaring fire).

Overuse of certain mechanisms (for example here ellipsis).

Meanders without any discernable goal from one scene to the next.

So in short, it’s a tale told by an idiot savant, full of sound and fury, signifying nothing.

Take this sentence as an example:

A biting incident, if I'm to be perfectly honest. Involving… several toes.

The ellipsis just gets in the way, the ‘perfectly honest’ likewise and this should be one better formed sentence. The several toes bit is overly explicit without being clear (he lost several toes, or toes were bitten?). Most of the text is like this - infuriatingly vague, clumsy in construction and full of non-sequiturs.

Compare this with real writing:

https://archive.org/stream/GrahamGreeneShorts/21%20Stories_d...

https://www.gutenberg.org/files/3077/3077-h/3077-h.htm#link2...

Did you read the original by Saki (Exhibit 12)?

Ideally, you should read all 14 exhibits. Each has been produced by a different LLM, some local, others frontier models of the time.

E4 = Claude

E6 = ChatGPT

E8 = Mistral

E11 = DeepSeek V3

E12 = Original

Sorry, I'm not doing all that work for you, I read the first one and was sick of parsing it for meaning by the end.

The work you posted was distinctly unimpressive, particularly as it had a seed text to impersonate and improve. The 'window' is I assume a french door, which I suspect is a mistranslation. If you want to seed LLMs I'd start with texts that are not translated!

Honestly compare it to the human writing examples I showed you and consider the differences in intent and style, which are vast.

I see why you're interested in this, I myself am interested in what these models can do and have tried similar experiments, but you should always be aware when you're asking others to waste their time on word generator output.

> The 'window' is I assume a french door, which I suspect is a mistranslation.

It is a french door/window. The original story is by Saki (HH Munro), one of the greatest short story writers of all time. He was English. The story is literally called "The Open Window."

> Sorry, I'm not doing all that work for you

Well, you asked. And I gave you stuff that I generated for comparative analysis. I am quite satisfied with the output. Most human writing, including genre writing, is subpar. LLMs are far superior to that. If you want to read only Faulkner and Eliot, then stick to them I guess?

> human writing examples

You must provide a solid prompt. And you must use a strong model. Deciding LLMs suck at writing based on output from Gemini 2 Flash, and expecting writing like Graham Greene is a bit much. The day LLMs start writing like him based on one-line prompts, writers will have no option but to look for other jobs.

In the past window could mean a glass door? I wasn’t aware of that, please forgive my ignorance. A minor point anyway and most problems were stylistic. I think it is clearer in the original work that it is more a door than a window in the contemporary sense, which is interesting.

I would not be satisfied with this output, I mean if your goal is mildly satisfying potboiler fiction I guess it could be ok?

I have read almost every kind of fiction under the sun. My standard for what is great is pretty high. This is why I am perfectly willing to accept LLM output as lot as the writer has put in sufficient work into the same. Because this kind of writing can produce middling to good enough fiction, like most genre fiction. Which I enjoy for the plot, characters and environment.

If a great writer used LLMs, then the output would be outstanding. Of that I have no doubt.

Reply on news.ycombinator.com