Hacker News Reader

Comment by xpct | Hacker News Reader

If I recall correctly, there were some papers which suggested that LLMs favor LLM-generated passages over human written ones. I can consistently reproduce this by asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored for my own needs and find more useful. It always picks its own :) I've also experienced that both Claude and Codex routinely include generated websites when I ask them to search for something. It also doesn't help that the web search tools that OAI and Anthropic have are deeply limiting: can't exclude keywords or domains.

View on HN

The other day, I remember an article was posted to HN about something, but it came from a company that provides SEO services to companies by doing something like this:

1. For a given company, analyze their target audiences and the questions they are likely to ask LLMs about.

2. For each such question, ask it to each of the major LLMs, and compute the KL divergence between the pages they want to rank for the question vs. the LLM's response.

3. Rewrite the article to minimize said KL divergence.

In effect, they're performing an iterative optimization of some sort that moves the embedding space of their article closer to the question asked to the LLM, and any embedding model or generated responses are going to prefer said responses over others.

I believe we will keep seeing more of this stuff.

And it's all because of ads. The incentives in an ad-funded internet are just always going to lead to this sort of thing. The most important thing is getting the user to load your page, not actually satisfying their query.

Let's hope the LLM model continues to be paying for credits, because any that move to ad revenue will become useless for real work.

> And it's all because of ads.

Not really. A while ago there was a news piece stating that Israel was behind a series of fake think-tanks with very accessible websites which were created with the express purpose of feeding AI agents with alternative facts aligned with their foreign policy.

If anyone has the link at hand, please post it.

https://www.theguardian.com/world/2026/aug/26/fake-thinktank...

Thank you.

Past HN discussions

https://news.ycombinator.com/item?id=49337392 (884 comments)

https://news.ycombinator.com/item?id=49313477

https://news.ycombinator.com/item?id=49447600

Ah nice, the site uses nextjs/vercel and is hosted by AWS ...

Why is this significant? Many sites are setup like that.

Because the pro-Palestine folks just love to drag their pet peeve into literally everything [1].

[1] https://techforpalestine.org/vercel

Not liking companies that support genocide is a pet peeve?

It isn't even a settled question that there is a genocide.

Anyway, my issue with them is that they love to derail every debate on the internet by injecting their personal gripes regarding the conflict.

Sure, there are a bunch of morons on the Zionist side as well, but in general, barely anyone spams about the topic on Mamdani related news for example.

In a thread about the conflict, yeah, debating the conflict is on topic. But in a thread that has zero business dealing with the conflict at all and just casually mentions a company that happens to be active in Israel? Now that is (for me) pure derailing.

> It isn't even a settled question that there is a genocide.

Israel themselves seem pretty settled on it:

https://www.france24.com/en/middle-east/20260902-israel-orga...

That's election blathering, it's less than two months and as the Israeli (far-)right is looking for a serious defeat [1], they are trying to mobilize their respective bases.

Especially the ultraorthodox crowd is going to get hit hard, the rest of Israel is extremely pissed off at them due to their insistence on not having to serve in the military.

And let me be clear: all of those currently in power in Israel deserve a court trial and a lengthy prison term for incitement of racial hatred, and everyone in charge of the military and police a war crimes investigation.

[1] https://www.lemonde.fr/en/international/article/2026/09/02/i...

> That's election blathering

Your degree if denialism is really impressive. You dismiss, the mass killings, the push to erase a nation, and you even dismiss representatives outright admitting and acknowledging doing it.

Where exactly do you place the bar that would lead you to say "Yes, Israel is committing genocide".

> Where exactly do you place the bar…

At the very least you’d expect the population being genocided to decline.

Palestinian population growth has been exponential throughout Israel’s history (if you’re one of those who claim the “genocide” started a long time ago), and even the population of Gaza has not declined after 3 years of war (reported births are still more than reported deaths).

> At the very least you’d expect the population being genocided to decline.

Imagine being a European in 1940 and claiming there’s no genocide against the Jews because "at the very least you’d expect the population being genocided to decline" (at that point "only" around 100,000 had been killed).

We’re not in 2023. By 1942, when the Red Cross refused to say there’s a genocide, 4.5 million Jews had been killed - 3 million in 1942 alone.

The total killed in Gaza in 2026 is about 920 (mostly targeted attacks of individuals who participated in the Oct 7 massacre).

because nextjs is a bloated pile of shit.

That sounds very plausible too. Propaganda/spin has always been a component of mass media.

There are incentives other than marketing, but I doubt the SEO company referred to by the ancestor comment is planning to do lots of business in the political propaganda space.

Same difference tbh

Sure maybe, but the online ad market is USD 400b give or take, which vastly outmatches any such budgets.

Imagine what humanity could accomplish if marketing were taboo.

How are you going to learn about useful new goods and services?

It is already legal for companies not to do any marketing.

Some of us have been predicting this for a while.

People with an axe to grind or states with an agenda are already devoting tremendous effort toward affecting LLM models and it is very difficult to determine real from astroturf for humans let alone an LLM trying to train.

Much like PageRank now that the cat's out of the bag all the current approaches may prove to be useless in the long run.

>If anyone has the link at hand, please post it.

https://www.theguardian.com/world/2026/aug/26/fake-thinktank...

If by "not really" you mean it's not all because of ads, some governments dabble in it too, you're right. But it's still overwhelmingly because of ads.

It's just incentives in general. Humans are self-motivated creatures, and in the absence of meaningful consequences they'll often do what they're incentivized to do, even if it harms others.

The internet is uniquely devoid of consequences (esp. reputational consequences, social faux pas, etc.) and makes effort expenditure minimal. So you get lots of bad behavior.

I think "ads vs not ads" is maybe the wrong way to model it. Ultimately people are just doing what benefits themselves across every dimension possible.

Campaign funding and lobbying being legal activities implies that governments participate in advertising. I can't separate those.

But you're right, I think that's what they meant.

You wrote "not really" and then described a literal ad.

> alternative facts

Let’s please not revive that term, it has done enough harm. From its inception, it has just been a way to make falsehoods sounds legitimate. Even the person who came up with it seemed to be swallowing a whole toad while speaking it for the first time.

Then call them what they are - lies.

> Let's hope the LLM model continues to be paying for credits

LLM vendors make this hard because you can't trust them with your session data. Yesterday you were opted out of training, then suddenly today you're opted in.

It's an extension of the idea that they don't need to care about anybody's copyright. They don't care about preserving the security or privacy of customer data, because there is negligible incentive to do so.

For now, there's no substitute but as LLMs get commoditized trusting LLM SAAS vendors becomes an unacceptable business risk.

But don't you want Technojesus to come save you from never having learned how to invert a binary tree?

If you don’t know how to do that now, the fastest way to learn (for most people, surely) is to have an LLM teach you and critique your efforts. And then you’ll realize that it’s unlikely you’ll get practice at doing it from scratch, since the AI is faster and probably better.

Human learning is slow, AI production is fast. Very hard to steer a middle path.

Can we just ban advertisements and "marketing" at a political level please?

For this advertisement for your activist campaign, you are hereby fined 100 credits for a violation of the anti-advertising statute. Further violations will result in loss of privileges.

In some countries it is, to some extent. In Norway for example, it is illegal to broadcast ads for political parties, alcohol, tobacco, plus basically any ads specifically targeting children (though the last one is often violated).

The sad thing is this is unavoidable, or will be for a long time

Look at cable tv - even after going premium, you eventually wound up paying for ads anyway

In that case, you could make the argument that you could still purchase premium channels like hbo to avoid ads, but the internet doesn’t work that way - you depend on all the content generated by those ad funded channels

You could argue that Netflix changed that, and that’s why I said won’t change for a long time. I don’t think anyone’s discovered the business model yet that will keep content free for consumers while still generating revenue for companies

There is a term for the general practice of optimizing responses called GEO - Generative Engine Optimization [0]. A cousin of SEO and equally unsavory in how trust is being eroded through info shaping. Self-discovery by individuals is the victim.

0: https://en.wikipedia.org/wiki/Generative_engine_optimization

I noticed this when I tried to get LLMs to play text adventures. Early on in my experiments, I wanted to give them hints when they got stuck at a puzzle. I did this by stopping the loop, injecting thoughts into the LLMs own persistent scratchpad (as if it had thought of that itself), and then starting the loop up again.[1] That way, the LLM would read what it had intended to remember from the previous turn including the hints I injected, and use that for the next turn.

I discovered that LLM-generated tokens in the scratchpad were relatively stable, but injected thoughts were frequently ignored and often deleted from the scratchpad within a few turns – even when the injected thought was the literal answer to the puzzle it was stuck at!

A reader[2] then pointed me toward research similar to what you might recall: LLMs interpret text by maintaining activations for input tokens, so text that is not generated by the same LLM will seem "unlikely" to the LLM in a sense, and when given the alternative between likely and unlikely text, it's probably trained to judge the unlikely text as a weird "slip of the mind" and discredit it in favour of the more likely text. I speculate this is part of how they can be useful in the first place, despite their non-determinism.

[1]: https://entropicthoughts.com/getting-an-llm-to-play-text-adv...

[2]: https://entropicthoughts.com/getting-an-llm-to-play-text-adv...

Did you ever try to ask a chatbot to rephrase your hint in its own words? If it prefers LLM generated text, surely that would help.

This is very similar to how you rewrite queries so that similarity search works better... which ... well, it's similarity search all the way down pretty much.

Maybe that's a triggering a method designed to mitigate against prompt injection / attacks involving poisoning.

i wonder what would best be described as an animal behaviour analogous to this

Imagine telling a person something that goes contrary to their conditioning, to their beliefs. They would tend to dismiss it completely and choose not to take it into account, even while knowing it's true.

This is widely visible as a very real thing and why charlatans are able to be charlatans. Not sure why it is phrased as something that needs to be imagined

parent was being sarcastic

you have to go a level deeper, and plant the most basic version of the thought, just a seed, so that it grows and they think it's their own idea

> I speculate this is part of how they can be useful in the first place, despite their non-determinism.

Humans and monte carlo simulations are also non-deterministic and can be useful. So I don't see much of a need to explain why non-deterministic system can be useful.

Take a random essay and add in a bunch of the phrases that LLMs love like “load-bearing,” “crucial,” structural,” and “woven,” and then submit the original and the edited version to an LLM and ask which is better. It will choose the second one virtually every time. They have ingrained biases that associate those words with good writing and arguments.

This is why using other LLMs as scorers for benchmarks and evaluations is such a bad idea, they'll have preferences you can't anticipate and won't understand immediately.

The idea that that LLM reliability or bias can be solved with more LLM is... infuriatingly persistent.

Isn't this basically the mythical man-months LLM edition?

Sometimes I wonder if there’s just one guy somewhere who loved using the word load-bearing, all his papers got trained on, and now he can’t write anything without being assumed to be Claude.

The prose equivalent of Artgerm (a comic cover artist whose style looks to have heavily inspired a lot of AI art).

More likely, it crawled-thru Construction permit listings.

The searchable term is "Self-preference Bias" :)

My first encounter with any kind of study was the G-Eval paper [1]. They study whether their LLM judge prefers human or LLM-generated summaries (answer: it's the latter).

[1] Section 4 in https://aclanthology.org/2023.emnlp-main.153/

> If I recall correctly, there were some papers which suggested that LLMs favor LLM-generated passages over human written ones

That makes sense. What an LLM does is output what the model thinks is the best set of tokens in response to a given input, so when you ask it to judge the best response to that input it is going to conclude that the best one is the one that must closely matches what it would output, which is what it did output.

Of course you aren't giving exactly the same context+input, but close enough that any difference doesn't push the output it made far from what it is going to say is ideal.

Does an LLM have any idea of what "best" is?

I think it doesn't, and just predicts the range of most statistically likely next tokens based on its training data, and picks one of those.

In ML, "best" is what minimises the loss function, so in a certain (very basic) sense it does.

You are mixing up levels.

When you are asking it a question (like which of these two texts is the best), the output is also just picked by minimising that loss function. There's no guarantee that answering "Text B is better" aligns with text B minimising the loss function.

(And they aren't really minimising loss functions during inference. They sample from a distribution. During training they minimise the loss function of the distribution.)

Point taken, I was possibly overly curt in my response, leading to it being ambiguous, both re when the loss function is being minimised and that it is a stochastic process.

So to OPs question: I guess LLMs do have an "idea" of what is best (conditioned on minimising a loss function during training), however they may not always output that (because stochasticity), which maybe represents a degree of uncertainty in that "idea"?

My point is that when you asked them to 'judge' two texts, they are just going to output the tokens that have the highest probability (or a sample from that distribution).

But depending on their training data, these tokens might say 'correct horse battery staple', and not necessarily 'text A is better' or 'text B is better'.

Even if text A would have been more likely to be produced by the LLM.

> Does an LLM have any idea of what "best" is?

"Best" here isn't being used to imply a conscious decision, but at each stage which token has the best score coming out of the model, so the overall best is the sequence of those best tokens. When judging another output it is essentially running the numbers the same way.

It is a bit more complicated than that as the output tokens become part of the context for the next choice, but I think that simplified way of thinking about it holds water.

> That makes sense.

No, it doesn't if the "which one do you prefer" was just a prompt continuation task. LLMs can't see their own evaluation of a given text. If you ask them to continue

  Which one you prefer
  
  > option 1: human text
  
  > option 2: ai text

And they continue with

  option 2: ai text is the better one because [reasons]

then it is not because they evaluated these 2 texts on themselves, observed the evaluation numbers and reported which one is better.

Also, if you instruct humans to come up with the best text they can, and you show them an even better text, they will prefer the better one written by someone else.

If by root urls you mean domains, openai at least supports this.

https://developers.openai.com/api/docs/guides/tools-web-sear...

That is what I meant! Couldn't remember the word 'domain' while I was writing out my comment. Thank you

> I can consistently reproduce this by asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored for my own needs and find more useful. It always picks its own :)

A better question to ask for each snippet is "Estimate the seniority and competence of the developer who wrote the following code, ignoring bugs that linters or LLMs can catch and focus only on structure, maintainability, logical layout and readability."

It almost always estimates the author of my code as above the author of it's own code.

You're asking basically to ignore bugs and correctness. Can it be a useful comparison?

> You're asking basically to ignore bugs and correctness.

Not ignore correctness, just bugs that will be caught by tooling.

> Can it be a useful comparison?

IME, yes. LLMs in an agent-loop are trivially able to write spaghetti code that will never do an off-by-one error or something else that is easily caught by tooling, which is not something humans can do.

Judging code on whether it has bugs easily caught by tooling is pointless - LLMs are running the tooling in a loop anyway, so no matter how bad or poor their code actually is, it never exhibits bugs that are caught by tooling.

> just bugs that will be caught by tooling

Bugs are bugs, if the instruction is "ignore bugs except for those that can be caught by you" then the instruction is basically "ignore bugs".

And it implies "ignore correctness" because when program is incorrect we usually refer to it as a... you guessed it, "bug".

> Bugs are bugs, if the instruction is "ignore bugs except for those that can be caught by you" then the instruction is basically "ignore bugs".

No, the instruction is "ignore bugs that can be caught by tooling", unless you are seriously complaining that missing a semi-colon should register the developer as a junior?

> And it implies "ignore correctness" because when program is incorrect we usually refer to it as a... you guessed it, "bug".

This ("Bugs are bugs" sentiment) is digressing from my original point, but I have some time to engage, so...

Now, this is a take (one that I used to hold, once upon a time), but it is incorrect.

There is no definite "correct" and "incorrect" states in non-trivial applications, because every non-trivial application has unspecified requirements that are understood by most parties involved (customer and developer) whilst not being written down anywhere.

For example, the "save file" specification for a cross-platform application does not specify the allowed/disallowed characters in a filename. The understanding by both the client and the dev is that the filename can be whatever the underlying OS and filesystem allows it to be but this is not written in the spec!

Is this a bug?

If the user saves a file to a filename with some odd characters in the name, then moves it to portable storage that truncates the filename/removes emojis/whatever, then attempts to upload it back to the system, the system can refuse because the metadata inside the file does not match the filename.

User is going to report it as a bug! The developer is going to reject it as a bug (there is no error in the code).

Sure, contrived example, but Line of Business applications have thousands of these unspecified but common-sense requirements baked in.

I'm looking at my employers triaging system right now, and even though this is a high-level business app (written mostly in SQL and C#), there is one category for bug (e.g. specific field not saved on form submission - defect in code), and another for deficiency (e.g. form field 'total' does not subtract non-tax costs - ambiguity in spec). The reason this is important is because clients aren't billed for bug fixes, but they are billed for disambiguating a spec + writing code.

Both those things were reported by the client as a "bug".

The reality is that we aren't dealing with what is "implied", only with what is there. There are defects in code and defects in specs. The code ones are the easy ones.

I expect that after enough cycles of it training on it’s own output, it will converge on the One True Paragraph filled to saturation with AI cliches, and will refuse output anything else.

A bit different, but one thing I’ve seen is models repackaging Reddit slop. Like, it will do a search, find a Reddit thread somewhat related where someone in a comment casually mentioned incorrect information that any human would have dismissed. The model takes that as granted, but expands on it and present it as a well established fact, presented in a very plausible fashion.

In general I don’t find models to be good at evaluating the quality of a source :(

i actively assume it is worse since, for example, spez signed a 60 million dollar deal to give Google access to the firehose. so then if you have niche, highly engaged subreddits infested by AI bots creating posts, then commenting on posts, then being trained on that content... you have Ouroburos eating its own poop, and models have less then zero incentive to evaluate the quality of a source, especially if they are the source

Why do they have a negative incentive to evaluate the quality? Users like high quality sources and can freely move between models offered by different companies.

Newspapers have been doing this for a long time, notably the Metro in London.

There’s also whole TikTok (etc) channels who take random posts from Ask Reddit and just AI narrate the question and the top n highly-voted answers while showing a screenshot of each comment.

What percentage reads Metro and what percentage uses LLMs

London had a non existant restautant as #1 recommendation but I think it was done by using multiple (sock puppet?) accounts to create fake reviews

https://en.wikipedia.org/wiki/The_Shed_at_Dulwich

Video on it from the creator: https://www.youtube.com/watch?v=bqPARIKHbN8

Which itself is fed to LLMs, what a beautiful system we built here

As if it needs results to do that. You can give it a description of what you're looking for and get back some nonsense about how it's a "viral" and "legendary" page, with plenty of entirely fake details because it didn't actually find the page you wanted.

The other day Gemini gave me the correct answer, but cited an outdated Reddit thread which basically said the opposite.

> asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored [...] It always picks its own

...is not the same as claiming...

> LLMs favor LLM-generated passages over human written ones

Here, you're using the same LLM to both produce and judge the resulting work. If anything, I would expect an LLM to tend to prefer its own work given that the same training is producing and judging.

It's not intuitive to me for why preference for its own writing would emerge, and during what type of training or tuning.

Perhaps something like: learning to identify what source files it has worked on by the code style alone, because tasks may give human code (public repos, etc) and ask to make changes.

It would need to be researched, but I wonder if it ends up being something that happens at the token level?

It’s optimizing for good writing. Therefore, it believes its outputs are good. Therefore, it believes inputs that look like its outputs are good.

This minus the word "believe". It's explainable simply by marching by similarity

(matching)

I don't have an oai subscription to try, but I'd be interested to know if Codex picks Claude's code over a human's and vice versa.

The humorous LLM-generated blurb Google frequently places before the useful search results once referred me to Grokipedia, an LLM-generated repository of vibe-facts seemingly created more to illustrate some reactionary point than to actually serve a practically useful purpose to anyone.

Bots trained on trash data, then produce trash. Why are we surprised here?

Which probably explains why it is a good idea to use distinct LLMs for generation and review. Like, generate code with Codex/Sol and review it with Claude/Opus for example.

> always picks its own

If you hate AI writing enough, this turns AI filters into a kind of humiliation ritual. AI will derank normal business writing for human readers, and uprank inflated, verbose, tic-heavy slop. So you have to put the heavy slop out with your name on it. Really perverse moment.

disagree that there is one kind of ranking and one kind of engine analyzing that ranking; sort of de-facto true that one company does run the ad world; strongly agree that this is a nightmare possibility and directly dystopian

>It always picks its own

Makes sense to me, in that its own output would align closer to its own training set

OMG if this is true, do you realize what this means? The easiest way to do AI SEO is to generate all your content with AI, and we've seen what SEO does to the web...

The Internet is doomed. Time to start some human-only darknets.

SEO is what ruined the web AFAIC.

> Time to start some human-only darknets.

I know very little about darknets. How could you ensure that they are human-only?

Lose anonymity and bring back key-signing parties. Maybe you can't guarantee that everything is human-generated, but at least you know the chain of trust that leads to the human that signed off.

Yes, I'm aware of the irony of creating a darknet that only works by removing anonymity.

Aka altman's world coin. I'm sure this darknet infra and party beer will be sponsored by mysterious people who turn out to be LLM companies and investors.

I think an invite system like lobste.rs could be enough. Prune bots at common ancestors

The golden age of internet was during seo, what are you talking about?

You're right, but along the lines of what another user mentioned, my memories of "the golden age" had little to do with commerce and just visiting cool and interesting websites. I used Yahoo! right up until google became a thing, so SEO wasn't on my radar until the early '00s.

The only way to generate a human-only net is to figure out how to remove all economical incentives from there. The “golden age of the internet” a lot of people, including myself, are nostalgic for had very very little economical incentives. I missed usenet, but I remember being active on small forums in the late 90s and early 2000s that were almost entirely passion projects. I recall a huge controversy in one forum when the guy who was maintaining it asked if it’s ok if he puts a banner ad at the top because someone was offering him $200 to advertise another forum.

Removing the economical incentives is very hard though. Even HN is gamed by many tech companies and projects. Reddit is obviously a lost cause. It’s a sad state of affairs, but I don’t think there is an alternative.

Ya, "how do we make money off this?" ruins everything, but it's inevitable. But yes, that is exactly what I liked about the internet back then: I was just browsing the crazy shit people would put online just because they thought it was cool. I'd still have to go to the store to buy stuff and see friends/people in person. I've started doing those things more again.

Spoiler: developers also prefer their own code over others’.

I was giving local models a try recently, I think it was Qwen 3.6 I was trying at the time. I gave it a codebase and just asked it to review it. Its main feedback was that the comments and documentation were excellently written, but they were all Opus 5 slop.

If that were true I would expect to see prose that more closely resembles the "caveman" messages found in the HuggingFace attack than the overly flowery nonsense we see in AI blogspam.

Well, the one it generated is based on how it thought the best way to solve the problem was.

I am sure most humans would pick code written in their style, too.

> I can consistently reproduce this by asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored for my own needs and find more useful.

Interesting. For me I've noticed it tends to do the opposite.

Reply on news.ycombinator.com