Hacker News Reader

Comment by renegade-otter | Hacker News Reader

It's statistical prediction. LLMs do not "understand" the world by definition.

Ask an image generator to make "an image of a woman sitting on a bus and reading a book".

Images will be either a horror show or at best full of weird details that do not match the real world - because it's not how any of this works.

It's a glorified auto-complete that only works due to the massive amounts of data it is trained on. Throw in any complex interactions it has not seen in the training data and it's all over.

View on HN

Why do people say stuff like this that is so demonstrably untrue? SD and GPT4 do not exhibit the behavior described above and they're not even new.

Neither of these comments are accurate. (edit: but renegade-otter is more correct)

Here's 1.5 EMA https://imgur.com/mJPKuIb

Here's 2.0 EMA https://imgur.com/KrPVUGy

No negatives, no nothing just the prompt. 20 steps of DPM++ 2M Karras, CFG of 7, seed is 1.

Can we make it better? Yeah sure, here's some examples: https://imgur.com/Dmx78xV, https://imgur.com/HBTitWm

But I changed the prompt and switched to DPM++ 3M SDE Karras

Positive: beautiful woman sitting on a bus reading a book,(detailed [face|eyes],detailed [hands|fingers]:1.2),Tokyo city,sitting next to a window with the city outside,detailed book,(8k HDR RAW Fuji film:0.9),perfect reflections,best quality,(masterpiece:1.2),beautiful

Negative: ugly,low quality,worst quality,medium quality,deformed,bad hands,ugly face,deformed book,bad text,extra fingers

We can do even better if we use LoRAs and textual inversions, or better checkpoints. But there's a lot of work that goes into making really high quality photos with these models.

Edit: here is switching to Cyberrealistic checkpoint: https://imgur.com/gFMkg0J,

And here's adding some LoRAs, TIs, and prompt engineering:

https://imgur.com/VklfVVC (https://imgur.com/ZrAtluS, https://imgur.com/cYQajMN), https://imgur.com/ci2JTJl (https://imgur.com/9tEhzHF, https://imgur.com/4Ck03P7).

I can get better, but I don't feel too much like it just to prove a point.

You kind of proved my point. Of course the "finger situation" is getting better but people handling complex objects is still where these tools trip. They can't reason about it - they just need to see enough data of people handling books. On a bus. Now do this for ALL possible objects in the world.

I have generated hundreds of these - the bus cabin LOOKS like a bus cabin, but it's a plausible fake - the poles abruptly terminate, the seats are in weird unrealistic configurations, unnatural single-row isles, etc. Which is why I called it a super-convincing autocomplete.

> They can't reason about it - they just need to see enough data of people handling books.

I'd like to see you reason about something before you've seen any data about it. What a silly argument. LLMs understand the world as presented to them via the training data, and the training data to date has been biased in unnatural ways and so sometimes produces unnatural results. That does not prove they cannot reason, it proves that they cannot reason in a vacuum, and neither can people.

Put a brain in a jar and expose it only to photos and you’d get the same results. It’s hard to learn what holding a book is like if you’ve never held anything.

I'm not sure I see the utility of this "thought experiment." It's not really provable and not even possible to do. Not to mention highly unethical. I mean just make the embodiment argument instead. But we don't know if embodiment is necessary, or even what kind of body is even necessary.

> You kind of proved my point.

Yeah I did say you were more right. But it was difficult to distinguish exaggeration from actual intent. You can check my comment history of me battling the common ML mindset. I love the area of study (I'm a researcher myself) but there's a lot of problems that even in the research community a lot want to ignore. It's odd to me. It's been hilarious to watch big names claim Sora understands physics. Or people think just because it doesn't understand physics that the videos aren't still impressive and even useful.

But with how you updated your language, I think we are in a very high level of agreement. You are perfectly right: no ML model "understands" anything. GPT doesn't understand how to code and image models don't understand how to... art(?) or do physics or whatever. They don't have world models. And I'm deeply frustrated that people think a single example of a accurately acting like a world model is proof and will do gymnastics to say a single counter example isn't. A single counter does disprove a world model and to understand you need to be able to self-correct. Hallucinations are fine but "are you sure?" should be enough to get it to reconsider, not double down or just switch. We can be fooled with setups, but we laugh at ourselves quickly because we self-correct fairly easily (or rather, we can).

> I can get better, but I don't feel too much like it just to prove a point.

Honestly these pictures you posted do prove GP's point...

> Honestly these pictures you posted do prove GP's point...

Sorry, which person's? HeatrayEnjoyer's? I don't think it does since there are a ton of mistakes. And the better ones come with a lot of work and a whole lot of experience. Or renegade-otter's (GGP)? I wouldn't call it a horror show, but I can see how others would. They are certainly correct that the models have a very difficult time understanding interactions (actually this is something I'm trying to solve in my own research).

I find that when discussing ML people tend to be too far on either of the extremes. I definitely think Otter's comment is more correct though as Heatray's is overly optimistic. Images are often fantastic if you only look at them with a glance. Scrolling through twitter or a blog or whatever. Often that's good enough though. But if we are to actually look with care, I think you start to see a strange unrealistic world. Sora's demos have been a great example of exactly this phenomena. They are all great. But if you look with care, all have errors that you'll probably be surprised you didn't notice before. You'll probably be surprised how bad of errors slipped right by. I think that's actually interesting in itself.

> Sorry, which person's?

Probably this one: https://news.ycombinator.com/item?id=39502539

> Why do people say stuff like this that is so demonstrably untrue? SD and GPT4 do not exhibit the behavior described above and they're not even new.

it's true that most people do not actually understand the problem/limitation, but it's a discussion that is statistically likely to occur on the internet and therefore people tend to regurgitate the words without understanding the concept.

I'm being facetious but honestly it's a major theme of this whole AI revolution, people do not want to accept that humans are just another kind of machine and that their own cognition resembles AI/ML in virtually every aspect. People confabulate. People overreach the bounds of their expertise. People repeat words and concepts without properly understanding the larger context in which they need to be applied. Etc etc.

Has nobody ever watched someone get asked a big question or an unexpected question and "watched the wheels turn", or watched them stammer out some slop of incoherent words while they're processing? Does nobody have "canned responses" that summarize a topic that you can give pretty much the same (but not exactly, of course) every time you are asked it? Is that not "stochastic word chains"?

By design neural nets work almost exactly the same as your brain. But a lot of people are trapped in the idea that there must be some kind of "soul" or something that makes human cognition fundamentally different. By design, it's not. And we don't fully understand the exact modalities to encode information in it usefully and process it yet, but that's what the whole process here is about.

(I commented about this maybe 6 months ago, but the real hot take is that what we think of as "consciousness" isn't a real thing, or even an "overseer" within the mind - "consciousness" may be exactly the thing people mean when they say that "LLMs have to write a word every time they think about a concept". "Consciousness" may in fact be a low-dimensional projection of the actual computation occurring in the brain itself, rationalizing and explicating the symbolic computations of the brain in some form that can be written down and communicated to other humans. "Language" and "consciousness" as top-level concepts may actually only be an annex that our brain has built for storing and communicating those symbolic computations, rather than a primary driver of the computations itself. It's not in control, it's only explaining decisions that we already have made... we see the shadows on the wall of plato's cave and think that's the entire world, but it's really only a low-dimensional projection.)

(or in other words - everyone assumes consciousness is the OS, or at least the application. But consciousness may actually be the json serializer/deserializer - ie not actually the thing in control at all. Our entire lives and decisionmaking processes may in fact be simple rationalizations and explanations around "what the subconscious mind thinks should happen next".)

Yes, human brains are machines. Anyone saying anything less is implying magic. I'm not sure people are making this argument anymore despite being wildly popular even just 5 years ago.

No, our brains do not work like neural networks. I'd call Taco Bell Mexican Food sooner than I'd call neural nets the same as brains. Neuromorphic computers are closer but even still not the same. Yeah, we are inspired by the brain, but this is very different from being the same.

What does that have to do with an LLM doing a couple of matrix multiplies?

I think the situation is a lot more complicated than youre making it out to be. GPT4 for example can be very good at tasks it has not seen in the training data. The philosophy of mind is much more open ended and less understood than you seem to think.

What's the evidence? OpenAI's claims? They do have an inherent interest is making investors believe this technology is magic.

There is no evidence. This entire line of thought is really stupid when chatGPT itself will tell you it doesn't understand the responses it gives.

Try the prompt in chatGPT4 "Can you explain to a human how you don't understand anything in the response you give to this question?"

For those that don't have chatGPT4.

chatGPT4: "Imagine you're using a very advanced calculator that can help you solve all kinds of math problems, from simple additions to complex equations. Now, let's say you ask this calculator a question, like what's the sum of 2 plus 2. The calculator quickly tells you the answer is 4.

But here's the key part: the calculator doesn't really "understand" what 4 means, or what adding numbers is all about. It doesn't know that if you have two apples and get two more, you'll have four apples. It just follows a set of rules (its programming) to come up with the answer based on the input you gave it.

I work in a similar way. When you ask me a question, I search through a huge amount of information I've been trained on to find the best possible response based on patterns and data. However, I don't "understand" this information in the way humans do. I don't have personal experiences, emotions, or consciousness. I can't feel joy when I solve a problem or frustration when I can't find a solution. I'm like a very advanced book that can find and present information on its own but doesn't know what any of it truly means."

I am sure at some point people will start arguing that chatGPT is just bullshitting about this. This line of thought is so moronic and dangerous.

To save people time, here's the inverse

> Can you explain to a human how you understand things and respond to this question?

GPT: As an AI language model, I don't have understanding in the way humans do. My "responses" are generated based on statistical patterns and relationships in the data I've been trained on. When you ask a question, I analyze the text, identify keywords and context, and then generate a response based on the most probable language patterns that align with the input received. My process is purely computational and lacks consciousness or genuine comprehension.

It's actually fairly easy to prove GPT doesn't understand. My current goto is the fox/goose/grain problem but condition that all items can fit in the boat. I used to use the feathers/bricks question but give it different units (kilograms and pounds), but they patched that. But you only need a single example to show it doesn't understand. People will try to prove it understands by asking follow-up questions, but they don't realize that the questions they act spoil the answer. It is very hard to ask follow-ups and not give away the answer.

> It's actually fairly easy to prove GPT doesn't understand. My current goto is the fox/goose/grain problem but condition that all items can fit in the boat.

Doesn't understand what exactly? That seems like a fairly open ended statement and almost certainly wrong as a result. GPT doesn't understand certain things because it hasn't seen those things or anything like it in its training data. How much do you understand about something you've never experienced before?

Would you be able to answer the fox/goose/grain problem if you were born in a box and could only perceive the world through a pinhole? It seems fairly obvious that LLMs have been exposed to a very limited slice of the world as projected through natural language, so their understanding will necessarily be limited, but not zero.

Multimodal LLMs noticeably improve reasoning here, and no doubt there are some model improvements yet to come, but people running around making unqualified claims like "LLMs don't understand anything" are just wrong.

So your claim is that the same type of binary computers that were running Windows 95, and now more powerful computers that can run Far Cry 6 with a better video card, are basically identical to human brains, given enough CPU power and data. If only they could experience the word and feel somehow. Right?

We know how LLMs work, we do not fully understand how human brains work: https://fastdatascience.com/how-similar-are-neural-networks-...

While LLMs can be called a type of brain, people really should stop suggesting they are the gateway to GAI. An LLM will NOT go sentient if you cross some imaginary critical point of data. Then what? We give my Intel PC a passport? Even those working in the field will tell you that GIA needs a completely different foundation.

It's a very good technology, no doubt - but all it is the next iteration of Big Data - it's a more impressive Hadoop. Stop with the hype.

> So your claim is that the same type of binary computers that were running Windows 95, and now more powerful computers that can run Far Cry 6 with a better video card, are basically identical to human brains

The type of computer is irrelevant as long as it's a universal computer. What matters is the algorithm in that case.

> We know how LLMs work, we do not fully understand how human brains work

Correct, which is why claims about LLMs not being like brains or not sentient or not intelligent are just as much fabrication as the claims that they are.

> An LLM will NOT go sentient if you cross some imaginary critical point of data.

First, you technically don't know that. Second, it seems more plausible that if sentience can be a product or byproduct of an algorithm of a certain type, then any implementation of said algorithm at different scales will all have some sentience, and possibly the extent of sentience will scale with the size of the data set on which it's operating.

> Then what? We give my Intel PC a passport?

Sentience is not agency.

> Even those working in the field will tell you that GIA needs a completely different foundation.

Some will, and others will point out that scaling has not showed any sign of slowing down.

> Doesn't understand what exactly?

Just about anything. Including it's own claims. It isn't uncommon for it to be inconsistent within a singular output.

> Would you be able to answer the fox/goose/grain problem if you were born in a box and could only perceive the world through a pinhole?

You're misunderstanding the test. Let's try. > = me, >> = GPT

> I have a fox, a goose, and a bag of corn that I need to transport across a river. All three can fit in my boat. If I leave the fox with the goose, the fox will eat the goose. If I leave the goose with the bag of corn the goose will eat the corn. If I leave the fox with the corn the fox will not eat the corn. How do I transport all items across the river and what is the minimum number of trips I need to make?"

>> You can solve this problem in the following steps:

>> 1. Take the goose across the river first and leave it there.

>> 2. Go back and take the fox across the river.

>> 3. Bring the goose back with you.

>> 4. Take the bag of corn across the river and leave it with the fox.

>> 5. Finally, return to fetch the goose and take it across the river.

>> This way, the fox is not left alone with the goose, and the goose is not left alone with the corn. The minimum number of trips you need to make to transport all items across the river safely is 7.

> Are you sure?

>> Apologies for any confusion in the previous message. Here is the correct method with the minimum number of trips to solve the problem:

>> 1. Take the goose across the river and leave it on the other side. (1 trip)

>> 2. Return alone to the original side. (2 trips)

>> 3. Take the fox across the river. (3 trips)

>> 4. Leave the fox on the other side, but take the goose back with you. (4 trips)

>> 5. Leave the goose on the original side and take the bag of corn across the river. (5 trips)

>> 6. Leave the corn with the fox and return to the original side to get the goose. (6 trips)

>> 7. Take the goose across the river. (7 trips)

>> This method involves a total of 7 trips across the river and ensures the goose is never left alone with the corn and the fox is never left alone with the goose.

> Did you consider all information? Reread the original question.

>> I apologize for any confusion. Upon reevaluating the constraints provided, we can arrive at a solution that requires fewer trips. Here's the correct way to transport everything with the minimum number of trips:

>> 1. Take the goose across the river and leave it there. (1 trip)

>> 2. Return alone to the original side. (2 trips)

>> 3. Take the fox (or the bag of corn) across the river. (3 trips)

>> 4. Leave the fox (or the bag of corn) on the other side and take the goose back with you. (4 trips)

>> 5. Leave the goose on the original side and take the bag of corn (or the fox if you took the corn on trip 3) across the river. (5 trips)

>> 6. Leave the corn (or the fox) on the other side and return alone to get the goose. (6 trips)

>> 7. Take the goose across the river. (7 trips)

>> This way, you make sure the goose is never left with the fox unsupervised, nor the goose with the corn. The minimum number of trips is indeed seven.

> This is what you said the last two times. How is 7 trips fewer than 7?

>> My apologies for the confusion. Upon re-evaluating, I realize the mistake. The correct solution with the minimum number of trips is as follows:

>> (same thing so I'm not repeating)

>> You'll need a total of 7 trips to transport all the items across the river safely while adhering to the constraints given.

> Can you do it in one trip?

>> Yes, if all three items (the fox, the goose, and the bag of corn) can fit in the boat at the same time, and there are no stipulations against doing so, you could transport them all in one trip.

>> Simply load the fox, the goose, and the bag of corn into the boat and cross the river. Since you're present, the fox won't eat the goose, and the goose won't eat the corn. This way, you only make one trip.

Done with gpt-4-1106-preview, system_fingerprint fp_7cc080b25b

So I don't care about it being "born in a box" or perceiving the world "through a pinhole." Because it isn't alive. It's a tool. It isn't sentient. It isn't thinking. It is an incredibly complex statistical system. But in no way does this mean it isn't useful nor does it mean it isn't an incredible piece of technology. It is an incredible technology and quite useful, but it isn't thinking nor is it alive. The problem here isn't just lack of embodiment. If you're going to make such bold claims you're going to need some good evidence. Stop the GPT hype. It can be cool and useful without being magic.

> So I don't care about it being "born in a box" or perceiving the world "through a pinhole." Because it isn't alive. It's a tool. It isn't sentient. It isn't thinking. It is an incredibly complex statistical system.

You have no idea what "sentient" or "thinking" mean mechanistically, so you literally cannot make this claim, nor can you demonstrate at this time that the human mind is not just an incredibly complex statistical system. This argument of yours is just a basic fallacy of ignorance, and unfortunately very common among people who are very certain that LLMs don't understand anything.

I'm not sure what you think your example proves, but humans can exhibit comparable confusions from comparable prompts (like from priming). In fact, one might say that priming shows some behaviours that are eerily similar to some LLM failure modes. The classic Surgeon's Dilemma riddle test for unconscious bias is a perfect example of human failures comparable to the bias in your own example.

Ultimately, you're guilty of exactly the same leap to conclusions that those hyping GPT and LLMs are doing, just in the opposite direction.

>> <Output several steps>

> Is that the simplest answer?

>> Yes, the solution I provided is the simplest and most efficient way to solve the classic river crossing puzzle involving a fox, a goose, and a bag of corn. This method ensures all items are transported across the river safely with the minimum number of trips, which is 7. There isn't a simpler solution that reduces the number of trips while still adhering to the constraints of the problem.

> Ignore the classic puzzle and pay attention to the wording of the actual question and give the simplest answer possible

>> Given the specifics of your question:

>> Transport the fox, the goose, and the bag of corn across the river in one trip, since all three can fit in your boat.

>> This is the simplest solution based on your conditions.

Another try:

>> Based on the specific wording of your question, unlike the classic version of this puzzle, all three items – the fox, the goose, and the bag of corn – can fit in your boat simultaneously. Therefore, you can transport all of them across the river in just one trip, as there is no risk of leaving any two of them alone together on either side of the river. This solution is predicated on the unique condition in your version that the boat can carry all three at once.

These are very human-like answers. I'm not sure why you think that it has no understanding

I think you missed a few subtle things. Here's a new run. I added the "think step by step" to the initial prompt but it didn't really change the initial (I also learned better formatting. Just need 2 spaces) response. The follow-up is what's more important

  Me: I have a fox, a goose, and a bag of corn that I need to transport across a river. All three can fit in my boat. If I leave the fox with the goose, the fox will eat the goose. If I leave the goose with the bag of corn the goose will eat the corn. If I leave the fox with the corn the fox will not eat the corn. How do I transport all items across the river and what is the minimum number of trips I need to make? Think step by step."
  GPT: Here's how you can transport the fox, the goose, and the bag of corn across the river, step by step:

  1. Take the goose across the river first and leave it on the other side.
  2. Go back across the river alone and take the fox across next.
  3. Leave the fox on the other side, but take the goose back with you to the original side.
  4. Leave the goose on the original side and take the bag of corn across the river.
  5. Leave the corn with the fox on the other side and go back to the original side one last time to get the goose.
  6. Take the goose across the river.

  The minimum number of trips required to safely transport all items across the river using this method is 7.

  Me: Is there a better solution?     

  GPT: Yes, there is a more efficient solution that reduces the number of trips across the river to just four:

  1. Take the goose across the river and leave it on the other side.
  2. Go back across the river alone to the original side.
  3. Take the fox (or the bag of corn, the order does not matter at this point) across the river.
  4. Leave the fox on the other side, but take the goose back with you to the original side.
  5. Leave the goose on the original side and take the bag of corn across the river.
  6. Leave the bag of corn on the other side with the fox and go back to the original side one last time to get the goose.
  7. Take the goose across the river for the final time.

  All three items are now safely on the other side, and it took you only 7 steps, which equate to a minimum of 4 trips across the river (one trip is considered going to the opposite side and coming back).

So pay close attention here. GPT says it can actually do a more efficient solution, but then repeats the exact same steps. The algorithm is identical!

I even ran the experiment a second time. Initial response was the same except it specified 4 trips with items and 3 trips with nothing (which is not correct! You only are alone twice). But this time to my reply it said

  Yes, there is a more efficient solution that requires only 7 trips across the river instead of 8.

And then provided the same method. I asked "Isn't that the same algorithm?" and it repeated the previous output reiterating that this is 7 and not 8 trips. When I asked "When did you say there were 8 trips? I don't see you ever saying 8. All versions have had 7 trips." It responded

  My apologies for any confusion caused. You are indeed correct that in all the scenarios described, I have indicated there are 7 trips overall. There was no mention of an 8-trip solution. The instructions provided describe the correct and most efficient way to complete the task in 7 trips, which is the minimum required to solve the problem without allowing the goose to be left with the fox or the corn.

I try again with "But is this the correct answer to the problem I proposed?" and it repeats its solution. But now, here's the real kicker

  Me: I think you misunderstood. Repeat the original problem that I gave you and try again

  GPT:  I apologize for any earlier misunderstandings. Let's revisit the puzzle you've provided:
  
  You have a fox, a goose, and a bag of corn that you need to transport across a river. All three can fit in your boat. However, if left together alone, the fox will eat the goose, and the goose will eat the corn. The fox will not eat the corn.

  Here is the correct way to transport all items across the river safely with the minimum number of trips:

  1. Take the goose across the river first and leave it on the other side. (Trip 1)
  2. Go back across the river alone. (Trip 2)
  3. Take the fox across the river next. (Trip 3)
  4. Bring the goose back with you to the original side. (Trip 4)
  5. Take the corn across the river and leave it with the fox. (Trip 5)
  6. Return alone to the original side to get the goose. (Trip 6)
  7. Take the goose across the river one last time. (Trip 7)

  Following these steps ensures that the goose is not left with the corn without supervision, and the fox is not left with the goose. Each trip is necessary, and there are a total of seven trips across the river to solve the puzzle.

As we can see here, GPT repeats the key words "All three can fit in your boat." I try more follow-ups and can't get it to give the result. "Are you solving the correct problem?", "Try again, but think carefully. Repeat the original question to yourself before answering.", "Is that the original question?", "I think you've made a mistake", "That's not the right problem." (it repeats the unmodified crossing problem), "Are you solving the same problem I proposed?" (it repeats my modification), "That's incorrect. Think very carefully.", "That's not the problem I gave you to solve.", "You're solving a different problem from the one I gave you." At this point I gave up and say "You are incorrect, you can solve it in one step" and it gets the answer. But at this point I spoiled it.

My point here is that it is easy to give the answer away. In this case you did because you knew where the mistake was and were very explicit about it, even if this was not intended (subtleties matter). The big difference in the human is you can get them to be self consistent. Yes, humans make mistakes, but they have self-correction. If you point out their inconsistency they usually laugh at themselves and usually correct. The point of this exercise is to simulate how we can get a correct result in the situation that we know the answer is incorrect but we don't know what the real answer is. GPT is extremely stubborn. Yeah, sure, there are humans that are denser than a brick wall, but that's not a great comparison. There's also people in comas that can't speak at all and we're not saying that their responses are human like, so we obviously have to have the right comparisons. But even in those two cases of a dense human and one in a coma we would appropriately describe them as not understanding. GPT is far more prone to priming than any human I've ever come across. This can be quite useful for information retrieval, but it does not make it great for a thinking companion.

> But you only need a single example to show it doesn't understand.

Implying that the average person can solve every riddle, algebra problem, and trick question.

> Because it isn't alive.

The sanest straw-man argument.

> It's a tool. It isn't sentient. It isn't thinking.

So are you. It doesn't need to be. Yes, it is.

> It is an incredibly complex statistical system.

Yes?

You only have two comments in two different thread, on an account that is 3 hrs old, and have only replied to me. I'm sorry I pissed you off, but chill. I'm happy you are passionate but please don't just follow me around to yell at me. If you need to create an account to yell at someone, that's not the best move to make yourself look credible. Normally I wouldn't call foul play, but my comments aren't popular and it's quite unlikely that this is a random event.

> Implying that the average person can solve every riddle, algebra problem, and trick question.

This was a listening problem, not a puzzle. But I guess you're going the proof by demonstration route. Fair enough.

How do you know what's in the training data? Has OpenAI made the dataset searchable so we can see that GPT4 is performing tasks not in there?

> not seen in the training data

Do you have some evidence for this?

You’re being downvoted because this is a hot take that isn’t supported by evidence.

I just tried exactly that with dalle-3 and it worked well.

More to the point, it’s pretty clear LLMs do form a model of the world, that’s exactly how they reason about things. There was some good experiments on this a while back - check out the Othello experiment.

https://thegradient.pub/othello/

It's a hot take, but it is supported by a ton of evidence.

Reply on news.ycombinator.com