Hacker News Reader

Comment by unleaded | Hacker News Reader

A lot of the replies are insisting AI will get better at writing with more development but I don't see it. Even if you have a mathematically perfect writing AI you still run into the same problems you would have if you handed off your writing task to someone on fiverr or something. It can't magically know what you want to say, it only has the information you gave it. A prompt complex enough where it won't get any wrong ideas has to contain as much information as the output would have.. so just write it.

View on HN

> A prompt complex enough where it won't get any wrong ideas has to contain as much information as the output would have.. so just write it.

This is only if the output is fully compressed. Writing is not just about encoding the writer's ideas but also about how the reader will ingest those ideas. The writer needs to consider when to put in rests in between complex ideas to help the reader flow through the text. This suggests the LLM could be prompted by a dense complex idea to be presented with the boilerplate needed for the human mind read smoothly and without unnecessary effort.

One of the biggest lessons in life to learn is there are no shortcuts.

Doesn’t stop people trying.

I also believe this. Post-training LLMs with vague metrics can only be achieved with RLHF, which is not impossible, but extremely costly and difficult. Instead, companies will opt for RLVR, focusing on math and programming tasks. This pushes objectives away from writing quality; often far away. That is why older models, in my view, actually read better than newer ones. It's by design.

You can brute force it by making it try random stuff then judge itself on it. You don't have to always use an LLM's output. Sometimes you can use that plus other things to add flavor. An LLM is actually really good at judging if something is good or bad. It just has a really hard time coming up with new things. But if you had unlimited compute you can throw in some rng and whimsy and get something resembling what humans do.

You can prove that doing this will spiral training into a fixed point. There was a lot of research into getting this to work in the past, but it never truly worked well. The hope was that if RLVR was used quite a bit, and the general performance crossed some threshold, that it would then be possible. However, since it has been shown that RLVR only concentrates the distribution of outputs rather than truly shift it, I doubt this will ever be a viable strategy.

> It can't magically know what you want to say

I think for this argument to be true, the axiom that supports it is that the models have just as much context as they will ever have, and you cannot see being able to give them more / enough to be able to understand your perspective. That feels unlikely to be a position that doesn't change. As a society we're giving more and more context each day to this, and that makes this a valid opinion now, but one that erodes over time.

Context went from 8,192 tokens on GPT 4 to 1M tokens currently with zero improvement. The latest models got even worse.

Size of context is not the entire story here, it's ability to properly feed and index the context that's needed on this sort of thing. E.g. your entire slack/discord/email/github/jira/zoom meeting/coffee chat ... history is the context that you bring to the table on this sort of thing. Most of this is unindexed. Much of this will not be in the future.

> The latest models got even worse.

Which models? This is one of those things that likely has both model and domain specific aspects that impact your experience. In my experience with OpenaAI models predominantly (I previously worked there), they've improved significantly over the last 6-12 months. My experience with Claude is worse, but I haven't spent as much time getting into a mechanical sympathy there. They're still not perfect though and I have many steering docs that help avoid the biggest problems in the models I use when generating docs.

Claude writing quality got unbelievably bad with Opus 4.7, with no improvement in Fable. Opus 4.6 was fine. Im starting to see it as a security risk - my brain just can't process its word vomit, so just tell it to go on, implement whatever

its not about dumping more and more info into the context, its about the intention. whats not in the context is just as important as what is. and i dont see how that can be automated.

also were seeing models become worse at writing as they get smarter.

Reply on news.ycombinator.com