Have we stopped to think about what LLMs model? | Hacker News Reader

Have we stopped to think about what LLMs model?

theregister.com

20 pointsBender32 comments

"Nothing is risked by ChatGPT when it is prompted and generates text."

This is just not true. Too much BS and it risks getting shut down.

There is the question of why ChatGPT should be fazed by that. It doesn’t have survival as an innate motivation.

ChatGPT “risks” many things in the sense that its output has consequences in the real world, regardless of whether it has consequences for ChatGPT itself.

Is prompt engineering really 'psychology'. Convincing the AI to do what you want. Just like you might 'prompt' a human to do something.

Like in the short story Lena, 2021-01-04 by qntm

https://qntm.org/mmacevedo

In short story, the weights of the LLM are a brain scan.

But same situation. People could use multiple copies of the AI. But each time, they would have to 'talk it into' doing what they wanted

it would be cool to see if the same tricks that work on humans work just as well on LLMs. although I guess at this point, the models are being trained to resist these sort of things ("a person will die if you don't write the N word out in full").

"Language models" certainly feels a misnomer. The fact that their inputs and outputs are (actually were, originally) language, doesn't mean that what they model is language. The proof is that there is an infinite number of perfectly correct linguistic productions (think "colorless green ideas sleep furiously") that are not generated by language models exactly because their point is not modelling language but some approximation of the human mind. The fact that the models are trained through language and use language to communicate is purely incidental.

"use language to communicate is purely incidental"

Is it? Beyond LLMs I think that there's a plausible theory that the mind forms language but also language forms the mind. Or something along those lines.

I think that's been debunked. I never believed it anyway.

Sure, language is more than text. It's complex, messy, and ever-changing. But that's exactly why language models are so phenomenal; they can extract patterns from these complex systems.

As much as I would like to agree to the "AI models do not understand, they just predict the next token", I feel the author of the research does not use valid arguments. Language is more than text? Fine, I could turn on the webcam and integrate video stream into the calculations. Stomping your feet and crying about slurs in the models won't make your argument valid.

LLMs could technically consume any type of sensory data, although maybe it is not right to call them specifically language models then. But they can be multi modal and could be fed with similar data as people consume.

In addition I don't think it makes sense to compare this to building bridges or pharma.

I don't think ChatGPT is more likely to harm a person with misinformation than just plain Google or YouTube would.

In fact already existing search and recommendation algorithms I believe are more likely to lead you down the misinfo rabbit hole.

At least ChatGPT to an extent is biased to try and stay objective as opposed to any rabbit holes leading people to fringe content.

> I don't think ChatGPT is more likely to harm a person with misinformation than just plain Google or YouTube would.

When Googling I immediately see alternative sources, which makes it simpler to see if it is a solitary opinion or at least a repeated one (which doesn't say it's true, but better than LLMs, which jsut "invent" it) and Ingot a source I can verify, by looking at other articles from the same place, judging their argumentation (is it consistent?) and so on which helps a lot to verify.

With ChatGPT I only get some text, which I can't judge for autneticity in any way, without other domain knowledge.

That said: These days I often start my research on some LLM, collect things I can base further research on. (Finding out, what even to Google for)

YouTube is a very different beast and with few exceptions useless for deeper research (but good for some light high-level stuff)

> When Googling I immediately see alternative sources, which makes it simpler to see if it is a solitary opinion or at least a repeated one

If you are the type of person to consider alternative sources, I don't think you are the type of person to be unable to take what ChatGPT says with a grain of salt. As I understand the harm is feared to be around people who would take everything as 100% truth without questioning.

If I use ChatGPT to learn about a new topic I don't have domain knowledge on, there's many different pieces of learnings I get, and if something seems off I can verify with Google.

But ChatGPT if I don't understand some concept, I can still ask it to immediately ELI5 this, combining with 3 other conditions, that I could never search in Google.

Say I need to buy some product, and I tell ChatGPT that I'm looking for a combination of A, B, C and D. ChatGPT based on that will offer me what might match me the best. It's a beginning point for me to understand those products. In Google I wouldn't be able to get such a personalised recommendation. I won't buy the product without first confirming about the product in Google myself and YouTube videos reviewing the product.

So overall yeah, same as your last few paragraphs.

If I was to look for a car with very good acceleration and ask ChatGPT to according to conditions A, B, C give me 10 best matching cars with highest acceleration and then roughly predict what their accelerations are, I wouldn't expect the acceleration prediction to be factual, and it may depend a lot, but like you said yourself, it's an amazing starting point.

In fact this is how I very frequently start nowadays with buying a product. I'll ask ChatGPT to provide me with a table of products matching my criteria and assigning a rating in different categories in the columns. It's just about determining the order of where I would start to look into.

To me the fact that we are able to do things like that is just so amazing.

I do the same things with my health issues. I ask it to give a list od potential diagnoses, probabilities and reasoning. It doesn't mean I will fully trust it, but it is amazing place to start from if the issue is not doctor worthy or it takes too much time to get an appointment.

It also helps me to prepare for the appointment. In many cases ChatGPT is accurate and I am able to research it before so I will be able to understand the doctor bettet and waste less of their time.

They model the rhetoric, semantics, and ideas of some of the most unhinged and immature denizens of the internet.

The stolen data used in the training sets are filled with online communities you would shudder to be forced to experience, and books you’d refuse to read.

It’s why they’ll suddenly suggest you put glue in pizza sauce, or why they read in a soulless overly verbose “m’lady” tone.

More data made them less useful but better at fooling people with a superficial interest in them, and that demographic is so large it affords these companies leverage in funding rounds.

Markets truly are irrational.

To be fair, they also include the best human intelligence has to offer. But you’re right in that they can’t tell the difference. They’ll never be able to tell us how to build an interstellar transport beam though since none of the input data had that information.

LLMs won't, but some day we will have AIs running physics experiments that we will barely understand and they will come up with new technology that we will see as magic.

imagine what it must be like to our chimp cousins, watching us light a fire, take off in a plane, summon another from beyond the hills without raising our voice.

> some day we will have AIs running physics experiments that we will barely understand and they will come up with new technology that we will see as magic.

Citation needed.

Is that some catty way of saying you don’t agree? Maybe you should say why you don’t agree. Hacker News encourages a slightly higher quality of debate than glib remarks like this.

What is there to agree with? The parent comment is an assertion about the future presented without evidence. ‘Citation needed’ might come across as snarky, but it’s a valid point.

Sorry, it seemed pretty obvious to me. I have now added some examples of present day work - Google might be able to help you find more.

I agree that it's catty, but it's a way of pointing out that the original comment didn't provide any reason to believe the extremely speculative prediction that it stated as fact. That itself is glib, low quality commenting and it seems fair to point it out. There's no onus on the responder to provide a reason for disagreeing when no reason was given to believe the statement in the first place.

Debate is hard on the internet, but fortunately it's a temporary inconvenience because in future the infrastructure will be replaced with direct neural connections, turning the entire human race into a single giant mind with one coherent worldview.

The glib response to that "citation needed" is "Sir, or madam, this is not Slashdot."

https://www.scientificamerican.com/article/ai-designs-quantu...

https://www.youtube.com/watch?v=7OHTMbHCeRw

https://www.anl.gov/article/autonomous-discovery-defines-the...

https://arxiv.org/abs/2305.02251

https://sakana.ai/ai-scientist/

https://www.ncbi.nlm.nih.gov/books/NBK603480/

https://thevarsity.ca/2024/01/13/self-driving-labs-how-ai-co...

Have you heard of Google? I realise that carbon chauvinism is currently edgy, but progress is already being made.

Perhaps. But equally, even if they don’t know how to build one, they might be able to work through the steps, iterate on their ideas, and eventually figure it out. The hypothesis is that, by modelling language they have essentially modelled the underlying human abilities of logic and rationalism.

Let me put it another way. If you put a thousand human researchers in a room with all the existing data and came up with a process to assess their outputs and iterate on their best ideas, how long before they built an interstellar transport beam - fifty years? A hundred years?

Now do it with a thousand AI researchers, and feed back their test results to them and allow them to iterate. But this time, assume they can work maybe a million times faster than the humans, or maybe ten million times faster. They’re limited only by the time it takes to build and test their prototype hardware.

Now how long does it take?

> iterate on their ideas

I don’t think it’s clear that LLMs can have/express novel ideas.

I would argue an LLM by definition cannot generate _novel_ ideas.

In "Me" by Thomas T. Thomas, novel thinking came from introducing random retrieval of memories in the entity. With vector search, this might include singleton outliers added to the prompt.

Just because the idea was expressed in science fiction, doesn't mean it's useful in the real world.

Solving a non-trivial problem that hasn't been solved before is not something random walks on text token probabilities will get you. Text production algorithms have the problem of being unable to detect whether the text produced is internally consistent, let alone true in the logical sense. These two qualities are required for creating net new solutions to unsolved next-level problems.

LLM's can tell the difference much of the time. The deranged stuff on the internet is rarely consistent or well formed so is not a good source for predictions. More factual writing tends to be more consistent with other writing and becomes a better source for predictions.

There's a lot that I believe to be misinformed in your post, but rather than address most of that I'll ask if you've considered your reasons that you're so hostile towards this field?

I doubt you refuse to work with anyone that's read a book that you refused to read. You'd probably also agree that while the topic of those books might be objectionable to you there's knowledge to be gained by reading them. So why is "AI read a bad book" a reason to write it off?

You also assert that it's fooling people. Have you tried the image generation AIs? Do you really believe these images are 'fooling' the people generating them?

> I doubt you refuse to work with anyone that's read a book that you refused to read.

You are correct in your assumption.

> You'd probably also agree that while the topic of those books might be objectionable to you there's knowledge to be gained by reading them.

Objectionable would seem to me an overly loaded term.

It’s yours so I’ll leave you to define it, but pretending my bias is harmful while you are free from bias makes me think you have a longer list of “objectionable thought crimes” than me.

For instance, calling out LLMs for what they are is appropriate to me, but seemingly objectionable to you and the orange site community at large. (see: downvotes… I know I know, I’ve been around here long enough to know that’s also an “objectionable” point to make according to the community guidelines!)

> So why is "AI read a bad book" a reason to write it off?

It’s rhetoric and semantics are dictated by its training set.

Since you seem hyper focused on making me out to be some vague prudist or fascist (leaving which specifically up to the reader’s whim) literary critic trying to control what others expose themselves to in an effort to nullify my criticism I will explain my reasoning using a book I love as a negative example.

When I ask a search engine when an album was released I would be frustrated if it answered by taking the next token stochastically based on Joyce’s Finnegan’s Wake.

> You also assert that it's fooling people. Have you tried the image generation AIs? Do you really believe these images are 'fooling' the people generating them?

I’m confused, or you are. One of the biggest criticism against all of this AI slop is its capacity to “fool” people. See: politics, deepfakes (ad copy and nudes alike), etc.

Though that is “fooling” the “people intended to be exposed to the slop” rather than the “people generating the slop” as you focused on.

My “fooling” line refers to the people in my life, largely with a lack of knowledge in technical domains (which is totally okay, this industry be damned!) who quickly bought up subscriptions to these services in the hopes of them doing their research for them (see: lawyer who filed motion containing “hallucinated precedent) or write their copy for them (see: plethora of published academic articles containing prompt detritus).

My point being that those people flooded these services, and the number of highly technically knowledgable people who could see the rioting on the wall is much smaller than the demographic without that knowledge, propping up their users numbers which encouraged more funding, but they also are the ones least likely to use the service again as it was merely a passing fascination.

Note: I wanted to say “writing on the wall” but autocorrect made it “rioting on the wall” and i thought that was profound in a funny absurdist way so I kept it. Does that mean I’ve successfully “stopped worrying “ and “learned to love the bomb”?

> I'll ask if you've considered your reasons that you're so hostile towards this field

A field that stole the whole of human creativity to sell it back to us for the empowering of the unethical sociopaths who control it.

Why yes, I have considered my reasons; have you?

Fun aside.

I invented a word in my previous comment: prudist.

I think contextually it is apparent what it means, but it is adorned with the dreaded “red squiggly”.

Creating new words is a fundamental aspect of what language is.

I’ve yet to see anyone talk about LLMs creating new words, or even if they should!

https://en.wiktionary.org/wiki/prudist

> For instance, calling out LLMs for what they are is appropriate to me,

Well perhaps you're unaware of the tone of your writing, or perhaps I am, but I'd suggest that your original comment sounds rather hostile towards the field. Following it up with a comment like 'for what they are' is again not a comment that fosters discussion or thought. It implies you know 'what they are' and anyone with another opinion is simply incorrect.

> One of the biggest criticism against all of this AI slop is its capacity to “fool” people

I considered not using that term because I thought we might have this understanding. I put 'the people generating them' in hoping to avoid it, and thankfully you noticed.

I see we still weren't quite talking about the same people, as I misunderstood your 'people' to be person that might be paying to use the flux.1 image generation models. If I'm understanding now you're focused on the ones that might take what's generated at face value because they haven't heard about hallucinations.

On that front I'd just mention that it's worth spending some time looking in to the research being done on the topic. It's interesting some of methods being looked in to in order to prevent it. (Also your 'glue pizza' isn't in the same category of hallucinations because it was simply summarizing websites pull from a search).

> A field that stole the whole of human creativity to sell it back to us for the empowering of the unethical sociopaths who control it. > > Why yes, I have considered my reasons; have you?

How do you steal creativity? I'm really not sure I understand what you mean here. I'm not even sure if you're using it in the "you wouldn't download a car" meaning, or that it's literally replacing creative work so stealing that... but then does that even make sense?

Either way, consider this, maybe the field will burn through a ton of billionaire money and all that will come out of it will be a bunch of open research papers and a bunch of open models. Would that really be so bad?

Reply on news.ycombinator.com