Comment by boesboes | Hacker News Reader

Comment by boesboes | Hacker News Reader

Parent Full thread

It's the natural evolution of content marketing & SEO. I work at an IT company where we have ~15 ppl building and managing a few SaaS apps for the travel industry, and we have 50-60 ppl doing marketing as a service for our customers.

They write page after page of content that barely get read. It all just exists to make the google bot happy. And that works. They bring in more customers & revenue apparently. (honestly, I wouldn't be surprised if it's all wasted and the added revenue is due to the better website and backoffice, but what do i know.)

So it seems only natural to have AI generate that content for the Google bot and stop wasting all this effort. Obviously I can't prove nobody reads this content, but seriously.. A landing page about hot-tubs and how cool it is to have one in your holiday home? Doubt that anyone actually reads that..

EDIT: I guess my point is also: it's already flooded with crap, just look at al the blog spam for things like 'how to start nginx'

  1. Good old days, humans talk to humans.
  2. Someone invents a bot, humans talk to bots.
  3. Someone invents another bot, bots talk to bots.

This is the story of SEO, banner ads, stock trading, consumer pricing.

There are probably many other cases.

The tech help spam: initially an attempt for a bot to talk to a human, but with LLMs crawling search results and aggregating answers, your answer to "how to start nginx" becomes the output of bot-bot interaction: an average of several generated answers (some initially and occasionally by humans).

The reason I can't stand ChatGPT answers is that it's trained on overly wordy SEO articles that need at least 500 words

Those 60 people can have a new job: fact checking the lies that the new ais produce

I think it confuses things immensely to anthropomorphize large language models. LLMs don't lie or tell the truth they just spit out text that is in alignment with the training model. Don't give them agency they don't have.

I don't lie either. I just spit out words that is in alignment with my mental model for optimizing my future.

“First you destroy those who create values. Then you destroy those who know what the values are…

But real barbarism begins when no one can any longer judge or know that what one does is barbaric.” — Ryszard Kapuściński

Quoting someone on the internet, saying: 'They are just liquidating their assets.'

Their lies are simple: they rule us.

(-;

Maybe you are a large language model?

Human social norms; definitions of optimizing for self, do not have to be maintained though; see the decline in religion.

Human society is not fractal; we die off, new social value stores come about; optimize for self could be defined as “not build environment destroying technology.”

A synthetic machine kept running but never allowed new inputs will just exist in long term belief in its old bullshit

Are you aware that what you are saying is not true?

If you are aware then it's a "Lie". If you are not aware then it's a "Mistake".

LLM's are not aware, so saying "lie" is not correct.

It’s a reasonable way to describe it. LLMs are designed to talk like humans so using human terms to describe them works quite well

Lying is intentional. A statement which is untrue is not by itself a lie, the thing which makes it a lie is the intention of the person.

AIs do not have intentions, the only thing in an AI is a statistical model of how likely words appear in certain contexts.

"Hallucinating" is a much better term, even though it still pretends a staristical model is akin to a human.

I guess that’s true. LLMs don’t really have intentions but there are some AIs that take actions to achieve a goal that can probably be described as having intentions

it's also not a great term for the same behavior in humans. There's the often quoted distinction by Harry Frankfurt about lying, which is knowing the truth but intentionally subverting or hiding it, and bullshitting, which is talking without any regard to truth or falsehood whatsoever.

It's a distinction worth making as people are much better at spotting lies than they are at spotting bullshit. The bullshitter may even be accidentally correct. AI models in particular will make up things in random places where nobody is sceptical or has their guard up because they see no reason for a lie, but bullshitters don't need one.

https://en.wikipedia.org/wiki/On_Bullshit

This is to fight back against the term "hallucinate" that everyone on here is using

What should I be calling the things that LLM's "make up" that simply don't exist?

That is an interesting point. I'm not sure why, but "hallucinate" doesn't bother me as much as "lie". Maybe it is because of all the ancillary baggage that "lie" has related to our current culture wars (lies, misinformation, disinformation, gaslighting, etc.)

I just asked (another anthropomorphic verb) ChatGPT for a better word and it came up with "fabricate".

> The term "fabricate" could be used to describe AI's tendency to generate fictitious information. This term has less anthropomorphic connotations than "hallucinate" and directly refers to the creation or construction of something, which could be a suitable representation for what an AI system does when it generates false data. Other alternatives could be "synthesize" or "generate," depending on the specific context.

I always thought that "confabulate" would have been the best word to describe what LLMs do:

> Confabulation is distinguished from lying as there is no intent to deceive and the person is unaware the information is false. Although individuals can present blatantly false information, confabulation can also seem to be coherent, internally consistent, and relatively normal.

> Can include autobiographical and non-personal information, such as historical facts, fairy-tales, or other aspects of semantic memory.

> The account can be fantastic or coherent.

> Both the premise and the details of the account can be false.

> The account is usually drawn from the patient's memory of actual experiences, including past and current thoughts.

> The patient is unaware of the accounts' distortions or inappropriateness, and is not concerned when errors are pointed out.

> There is no hidden motivation behind the account.

https://en.wikipedia.org/wiki/Confabulation#Description

Confabulate is more of a neurological term so I think that's why it never caught on, but I've always thought that was the perfect word for what's happening

"Fabricate" has the same problem as "lie" and "hallucinate". All of those terms imply a cognition that isn't actually happening.

I think the best way to refer to these things is the more accurate "error". The LLM isn't lying, it's in error.

It doesn't. Hallucination in this context means 'an unfounded or mistaken impression or notion'.

There's one side of the current zeitgeist that over-anthropomorphizes these models. But there's another side that seems to be terrified that LLMs could be anything resembling intelligent

Both sides are pretty emotion over facts though. A lot of their "human-like" behavior is emergent from things that don't work the same way in a humans... but we also don't understand cognition enough to then say there's no overlap between what LLMs are doing and what some part of our own thought process is like. If anything we have more studies that imply the opposite going back decades: https://www.sciencedirect.com/science/article/abs/pii/S09266...

> It doesn't. Hallucination in this context means 'an unfounded or mistaken impression or notion'.

Correct. Which, as near as I can tell, isn't the correct characterization of what's happening with LLMs.

> But there's another side that seems to be terrified that LLMs could be anything resembling intelligent

I certainly have no such fear. I say that only to indicate that my biases are not of that sort.

The first sentence seems contradicted by the second no? If you don't have that fear then why bother playing around the semantics of "impression or notion"?

Is it an unfounded or mistaken cluster of tokens which resemble an impression or notion then?

At the end of the day it's producing an output which systemically was intended to be accurate, but ended up not being accurate through a mistake in generation: That's a hallucination.

I think "fabricate" is much less aligned with cognition.

It is completely reasonable to say that "This machine fabricates widgets." or "This algorithm fabricates pseudo-random song lyrics." without being worried that the machine or the algorithm is exercising "cognition".

Yes and when you’re using “serverless” services there are actually servers running somewhere. At this point, it’s not insightful for someone to mention either on HN.

If we are going to expand the discussion to general inaccuracy in human communication we are going to need a much larger thread.

I do agree with you that "serverless" is a misnomer and not a useful technical term. I'm guessing the marketing team wasn't fond of "dynamically provisioned and auto-scaled pool of virtual machine instance-based services".

So then should we also get rid of terms like “bugs in code”, “viruses” as it relates to computers, etc?

In a technical forum, most people have the domain specific knowledge and the “well actually” is not actually needed.

Absolutely. The lie is in providing the false framework, or a framework with fakery embedded. If the lie is accepted at the framework level, individuals will believe themselves to be saying the truth, unaware that they have already swallowed the larger falsity.

Why would they? Google sees more content so you now have more nodes connected to you, so you appear higher. Whether it's true or false doesn't matter to Google.

This seems similar to, maybe even a restatement of, the "how do you detect AI-written content" problem.

If someone could provide a way to programmatically detect accurate, high quality content, it would solve the SEO spam problem. It would also as a side-effect solve the AI-content problem. Or maybe it's the other way around and it would solve the AI-content problem and as a side effect the SEO spam problem. Well, they'll soon be the exact same thing, and either way I'm not sure it's solvable.

The detection needs to be almost perfect for it not to be worth it for the AI to simply retry until it's passed the detection.

If an article is posted to a blog, and nobody is there to read it, does it even matter if it lies?

What they want is to optimise the ranking Google gives them. If Google's ranking filters out falsehoods then lies can be optimised away automatically.

Jobs to fact check the lies profit motivated humans generate should already exist.

They can become AI-empowered content creators. Their own Miyazakis, Scorseses, and Eilishes. [1]

[1] This isn't sarcasm. I think that's the world we're headed to. Hundreds of thousands of little creators serving their own unique niches. A world without Disney.

Reply on news.ycombinator.com