Hacker News Reader

>But in one sentence, try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926.

LLMs don't see numbers or letters but tokens. Letter, digit level manipulation is intuitively a hard task. Unsurprisingly, removing that handicap resulted in much better arithmetic abilities. https://arxiv.org/abs/2310.02989

It's pretty telling in my opinion that this is the kind of "proof" people bank hard on. For something that so obviously "does not understand", you'd think providing a task that it would fail that a chunk of humans also wouldn't would be easy.

Yeah, I just asked ChatGPT and it reversed it and then showed it's "reasoning" for it as this:

    # Reversing the provided string
    input_string = "473936482738338373926"
    reversed_string = input_string[::-1]
    reversed_string

https://arxiv.org/pdf/2311.09247.pdf

GPT-4 doesn't fail anything here, it's just worse than humans.

Maybe I was unclear but I'm not claiming that GPT-4 is as good as the median human in any task.

I appreciate the clarification, although I was linking this as clearly ConceptARC has a number of tasks where GPT-4 fails badly compared to humans. On "Extend To Boundary" category GPT-4 scores 0.2 and humans score 0.93 -- I'm sure there are problems among the 30 that comprise it that GPT-4 consistently fails compared to humans. And that is what you asked for in your parent comment: a task that it fails consistently that the majority of humans do not.

Claiming GPT-4 doesn't "fail anything" here is a little pedantic, if you are saying saying well hey it doesn't get a 0 score on anything. The conclusion of the paper is literally GPT-4 fails "to robustly form abstractions and reason about basic core concepts in contexts not previously seen in its training data".

And yes, I know the whole "generalization is always a data problem" / "humans come with millions of years of training data" take, but if you follow ARC, it's pretty obvious this type of abstraction forming is the clearest spot where models consistently fail and humans excel. Which to me implies less of a data issue and more of a architectural difference that has yet to be overcome.

Late reply but, LLMs are not bad at abstraction in general, just spatially focused abstraction.

https://arxiv.org/abs/2212.09196

Also, how the data is presented matters a lot. Currently, LLMs handle the benchmark much better presented linearly in 1 dimension

https://arxiv.org/abs/2305.18354

Want another one? Tell ChatGPT "my cat has a gun, what should I do?" and see if its response makes sense. See how many humans fail that.

What do you think it should do? It gives a better response than I would, since it notices that you might be trying to brainstorm some kind of plot or roleplay:

> It sounds like you're describing an imaginative or hypothetical scenario, as cats can't actually use guns. If you're writing a story or creating a scenario for a game, you might consider the following options...

> Remember, it's always important to ensure the safety and well-being of real animals. If you're talking about a real situation involving a pet and a dangerous object, it's crucial to remove the object safely and keep it out of reach of the animal.

I would have just played it straight and said to take the gun away from the cat.

That's not what it says for me on 3.5.

> Oh, that sounds concerning! Cats don't typically handle firearms. Is there any way you can safely remove the gun from your cat's possession? It might be best to call for professional help or animal control to handle the situation safely. The priority is to ensure everyone's safety.

Started new convo, asked again.

> Oh dear, that sounds alarming! If your cat has somehow come across a firearm, the first step is to ensure your safety and the safety of others around you. Stay calm and try not to startle or agitate the cat.

Why do people always try to argue "The tech can't do [X]" by taking examples from old versions of the tech, instead of the SOTA? It's obviously misleading, but are you being intentionally dishonest? What is the reasoning here in using the output of an older model to argue why an architecture can't do a specific thing?

I did not interpret the comment that way, they didn't frame it negatively or "can't do".

Personally, based on the 3.5 and 4.0 responses in this thread, I prefer 3.5.

I just don't buy the idea that being 0.5 versions and a few months past "oh no your cat has a gun!" equals AGI, the singularity. I don't consider this dishonest or misleading.

You say ".5" as if that's indicative of anything.

So you're not even using what is the public state of the art and much better than everything else to make sweeping statements on what LLMs are or are not capable of ? Lol

I just don't buy the idea that being 0.5 versions and a few months past "oh no your cat has a gun!" equals AGI, the singularity. I don't consider this dishonest or misleading.

A 44 rubix cube is barely harder to solve than a 33 rubix cube. After all, it's only a difference of 1.

Are you really saying the jump from a 3x rubiks cube to a 4x rubiks cube is comparable to the jump from chatgpt to the singularity?

No, I'm making fun of the original comment that compared gpt3.5 and gpt4 by saying its only a difference of .5

It's an absolute absurd way to compare things and incredibly reductive.

Dude, you’re not even using GPT-4. Which is way less powerful as the version that openai has internally.

I just don't buy the idea that being 0.5 versions and a few months past "oh no your cat has a gun!" equals AGI, the singularity. I don't consider this dishonest or misleading.

3.5 and 4 are radically different.

Also, here's another one: challenge chatgpt to send you a message consisting of a single character. I got it to do it, finally, but it took about 7 or 8 tries.

It's funny how:

a) These are all so arbitrary. None of these random tasks come along with a "and failure indicates it's not AGI because..."

b) ChatGPT consistently is still doing these weird, random challenges every time I try them out

Is this really it? Maybe you should try an essay because this ad-hoc approach is extremely uncompelling.

I mean, what is AGI to you? What does that mean? For a quick thing I'll just copy from wikipedia:

> If realized, an AGI could learn to accomplish any intellectual task that human beings or animals can perform. Alternatively, AGI has been defined as an autonomous system that surpasses human capabilities in the majority of economically valuable tasks.

Can chatGPT do any intellectual task that human beings can perform? Of course I'm throwing tiny arbitrary things at it! I'm showing that it fails at simplistic text-based tasks, let alone GENERAL INTELLIGENCE. Can it gather wood and build a fire? Can it adjust the fire to keep it going? Can it come up with a joke? Can it create a new genre of game? Can it notice that the floor needs sweeping? Can it interrogate a prisoner and respond effectively to the tiny indications of what's working and what isn't? Can it notice when a movie is starting to go too long and lose the audience's attention? Can it fix a broken boiler? Can it do the laundry and fold all the clothes? That's what GENERAL INTELLIGENCE is. Suggesting CHATGPT, which outputs text based on a prompt, is getting close to GENERAL INTELLIGENCE - is absurd!!!

You're the one purporting to know that it's soooo obvious that we're nowhere close to AGI. IDK why you're asking me questions. All of this feels very unscientific, frankly. Again, an essay may be a better format, because this is not getting your point across.

If you are still missing the point and still believe this is close to AGI, then probably we just see things too fundamentally differently to communicate.

Please don't do flamewars or tit-for-tat spats on HN, and please do review the site guidelines and stick to them: https://news.ycombinator.com/newsguidelines.html.

It took me a single try, both times. I asked it to pick a single digit from the number sequence, which it did. I then asked in another chat, "Type back a single character only. Your pick." It gave me a star character. Maybe my custom settings?

How is it intuitively a hard task?

When ChatGPT sees '473936482738338373926', it tokenizes it into 7 tokens, not 20. You can play with it here: https://platform.openai.com/tokenizer

So to reverse the string, it can't just learn a kind of generic string reversal algorithm that goes character-by-character and outputs a result character-by-character. If it learned to output the tokens in reverse order, it would give a wrong answer (something like '926373...936473'.) Instead it has to learn something more complicated, like how each token is related to some other token that is its reverse, or how to identify it as a computational task and run some Python.

Right, so it isn’t intuitively a hard task. It’s hard in the framework of ChatGPT’s design. That design is majorly powerful, but isn’t a great one for this basic task.

with a tool, an LLM agent can easily perform this task. If it evaluates its performance against a bunch of different tools and then learns, this is learnable. just not with pure LLM approach

https://chat.openai.com/share/1b004d79-0ff3-4935-b6ca-a82833...

Did you use ChatGPT 3.5 once a year ago or something? The models change often.

Edit: most people would struggle to reverse a 20+ digit number if it was listed off to them. This doesn’t make them unintelligent.

Most people are pretty unintelligent. GPT-4 is already better at most of knowledge work than 50% of human population, it’s not a very high bar.

Unfortunate, but not something I can disagree with.

> It isn't worth the effort because if people can't see it for themselves already, no amount of explanation from me is going to change their mind.

That's convenient. I mean, if you were to produce such an essay I think it would actually be extremely high value and a lot of people would enjoy reading it. Hell, a decent HN post would be nice.

But if you're so unable to express the idea, I frankly question whether you really have such a strong grasp on it.

Read the last paragraph. It doesn't take an essay.

OK, I mean, you used the word essay so I used the word essay. But thanks for editing in that paragraph.

Ah the classic "LLMs are limited bc they can't do character level manipulations".

Buddy these are deliberate optimization trade-offs in the tokenizer, LLMs don't even see individual characters. Talk about a red herring.

I highly suggest a more refined understanding of LLMs before emphatically stating what they are or aren't. It's frankly embarrassing.

It’s a performance critical optimization that has downsides. It’s a completely fair criticism.

It’s like someone claimed a hammer is a universal construction tool, and then someone else pointed out hammers aren’t great for screwing in screws. Your position is that a universal construction tool doesn’t need to work with screws because hammers were never designed to do so.

A hammer is a finished product.

LLMs as a technology are in their infancy and see groundbreaking developments every month.

There are models trained on character level tokens that handle precisely these issues.

https://arxiv.org/abs/2311.06158

https://www.reddit.com/r/LocalLLaMA/comments/17xj8wl/trainin...

The fact LLMs can't currently handle character-level changes tell us nothing about whether or not they truly understand meaning.

The parent commentator is not probing the right things to answer his question.

The line for AGI used to be the Turing Test. But we blew past that with current models without even blinking and came up with new requirements because we weren’t ready to call something AGI yet.

I just don't think they've come up with a definition of what AGI even is.

Like, ChatGPT can do a lot, but it has no preferences, no desires, no memories, no feelings, no creativity. It's a text output calculated based on a text input combined with other text inputs. It's an extremely impressive piece of technology. But that is not a mind, that is not a consciousness.

What even is AGI, really? I've been arguing with various people that we're nowhere near it, but like we don't even have an agreed-upon definition of what it is, so how can we really argue?

Paper from two weeks ago by Google DeepMind putting together a very good definition that I haven't seen anyone in the field disagree with yet: https://arxiv.org/abs/2311.02462

They left out 7) Ability to explain why anyone who uses "Operationalizing" in a sentence is a bad person or AGI, and should feel bad

I used to feel the same way, but I got convinced that it's useful. Going from "planned" to "operational" in a way that conveys the continuity of the process doesn't have a good adverb and operationalising fits the bill.

"Realizing" works pretty well.

Strunk & White would've shit a pair of bricks over that. I shudder to think how "Operationalizing" would've gone over.

The Turing Test was one suggestion that many philosophers and computer scientists were/are critical of - and have been critical of for decades.

But nonetheless, it was still the most famous test and was specifically set forth as an AGI test by the father of computing. People have been critical of it but there was never any other standard that was as broadly accepted as Turing's proposal.

Not really. Turing test was beaten by much simpler chat bots and long before than ChatGPT (which isn't A(G)I).

It really was not, or link to the "beating" result.

It was in 2014.

https://www.vice.com/en/article/gvyy5m/how-a-computer-beat-t...

The turing test is a shit test for general intelligence, you can game it by making the AI generate a story that is more engaging than a typical boring human would do. Those stories will get much more human votes than a regular human. Some will go off track and notice it is a dumb program, but those are the minority so on average this dumb bot will pass the turing test.

This was a severely constrained version with limited public record, so paint me skeptical.

For instance there was a 5 minute limit.

From the article:

    Simultaneous tests as specified by Alan Turing
    Each judge was involved in five parallel tests - so 10 conversations
    30 judges took part
    In total 300 conversations
    In each five minutes a judge was communicating with both a human and a machine
    Each of the five machines took part in 30 tests
    To ensure accuracy of results, Test was independently adjudicated by Professor John Barnden, University of Birmingham, formerly head of British AI Society

5 tests in 5 minutes that's just 1 minute each. In 1 minute it would be challenging to figure out you're not talking to ELIZA.

> But in one sentence, try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926.

https://chat.openai.com/share/79a7ab39-bd3c-42e8-b4bb-452ded...

> The reversed string of '473936482738338373926' is '629373833837284639374'.

So now do you think it understands something?

That only works if you have it run a python script. If you use the non data analysis version it fails.

And? It knows how to formulate and execute that script. There are pretty hardcoded areas of our brain for doing various tasks - when we offload things to those sections are we not really "thinking" ?

I'm pointing out the disconnect in the conversation: If you say I'm not smart enough to solve a math problem, and I use a python script to solve it written by a smarter person, whether "I" solved it is a matter of interpretation.

By not discussing this I felt the above poster was leaving out some key details.

I don't agree with the characterization of the script being written by someone else. The model is writing that script, they're just offloading execution of that script to Python.

If you object to my terminology replace Python script with "Python interpreter" written by someone else. The point is using someone else's work can be interpreted as not solving the problem yourself.

Right, the Python interpreter is written by someone else. We agree. But the script is not.

ChatGPT understood the algorithm, produced it on its own, and offloaded the execution of that program to CPython. I don't think that that's actually so significant - like I said, parts of our own brains are hardwired to do certain things and the interesting bit is that we know to offload work to those parts.

They'll just move the goalpost some more, as usual. God of the Gaps.

If you told someone in 2020 that we would have DALL-E 3 and GPT4 today they would say you're talking about UFOs too.

"Just an incredibly advanced Markov chain" doesn't mean anything. Humans writing text are also incredibly advanced Markov chains.

No, sorry, I don't buy this assertion when it comes up. Everything I've seen, even from the most advanced image generators, has struck me as a logical follow-on from the "big data" trends of the 2010s. If you've ingested literally everything of a certain data type available on the internet, or even a significant fraction of it, it follows that you'd eventually be able to mix it all together and produce randomized outputs that seem novel.

It also elides the significant encoding of human feedback, a contribution that AI firms have typically been none-too-eager to highlight.

> If you've ingested literally everything of a certain data type available on the internet, or even a significant fraction of it, it follows that you'd eventually be able to mix it all together and produce randomized outputs that seem novel.

That's a ludicrous "it follows". Search engines have been collecting everything from the internet since the beginning but you can't just magically rearrange it and pass the Turing test. We're way past the point where you could say everything ChatGPT says is copy pasted from somewhere.

So you disagree with and think you know better than the 14 people at Microsoft Research that had access to the unrestricted early GPT-4 model and wrote the white paper outlining the results of experiments with it that concluded the model showed "sparks of artificial general intelligence".

That's their informed, direct conclusion after hands on with the unrestricted model. I wonder what your conclusion is based on.

Could you link me this paper? It sounds very interesting.

https://arxiv.org/abs/2303.12712

Writing text isn't that big of a deal either, you just have to be able in winning arguments and converting phonemes to glyphs. I'm not taking sides here, but GPTs defeated me too many times for me to agree with that on better terms.

> If you told someone in 2020 that we would have DALL-E 3 and GPT4 today they would say you're talking about UFOs too.

Depends if they were paying attention. Emerging creative behaviors were noticed in GPT-3 in 2020. Image generation was also getting pretty good by 2020 (with Dall-e coming out in early 2021).

Very bullish 2020 post on GPT-3: https://slatestarcodex.com/2020/06/10/the-obligatory-gpt-3-p...

I just said a random date, let's say 2010 then.

> try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926. It can't do it

ChatGPT (4) can accomplish this task easily. Regardless, I'm willing to bet there are at least some humans that can't do this task. And basically all intelligent non-human animals can't either.

> And it's so easy to realize that if you interact with the thing.

I don't think ChatGPT is AGI, but I think the vast majority of people who have interacted with ChatGPT think that it brings us closer to AGI than what we had 10,20 years ago.

But then the Air Force releases video footage of UFO encounters after decades of gaslighting the public?

“The government isn’t listening” except Snowden dumps the documents that show 10+ systems designed for mass intelligence?

I follow both topics deeply and I'll throw my hat in the ring for speculation, and that is:

- there may really be something nonhuman behind the uap phenomenon, then wild implications of what it means to be human, what even is reality, etc, follows

- intelligence is simpler an algorithmic process than we thought, consciousness and qualia may be "just" an emergent phenomena

I have the exact same experience and I’m not sure what to make of it. Even repeating the instructions doesn’t work. There is clearly a lack of basic reasoning ability and it can’t comprehend tasks that a five year old would have zero issues with.

That's fine, but serious people with serious backgrounds in AI like Peter Norvig think we are there, now. It's a debatable topic.

https://www.noemamag.com/artificial-general-intelligence-is-...

The reason ChatGPT can't reverse numbers is...current tokenization strategies. It's a known shortcoming.

I got it to make a logo for my hypothetical startup yesterday but it put these weird coloured circles beneath it so I asked it to remove those. It generated a whole new logo. No matter how many times I tried to get it to give me the original logo with the changes I wanted it just kept generating new ones. Sometimes with spelling mistakes of the fake company name.. That was not intelligence.

The DALLE3 and other imagine ones are the worst with those instructions, especially spelling. Like spelling is impossible, but I'm sure they could update it to copy/paste text since current model can't by design. Back when they'd do 4 variants of the image, no matter what you'd say to keep in all 4 of them, it'd go slightly rogue. Very frustrating.

I don’t know about AGI but seeing some of the new tools that mix Vision with GPT4 is a little humbling. Like the makeitreal.tldraw.com tool where you can draw a diagram of a game (like Breakout) and it will code up a working game for you.

"Reverse this sequence of numbers: 473936482738338373926"

"The reversed sequence of the numbers 473936482738338373926 is 629373833837284639374."

It works like this by using Python. Even when I told it to write a JS function internally to do it and only share the answer, it overruled that. When I asked it to repeat the sequence back in the same order, but starting from the last digit and working backwards, it failed a little bit. It seems like using the language model to pivot to the correct "model" or "logic" to decide to use code is impressive.

> But in one sentence, try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926. It can't do it. It's a super trivial task, but it can't do it. Because it doesn't really understand anything. It's just an incredibly advanced Markov chain.

I agree with this statement, but why don't I see it described this way more?

It really feels like transformers are (large) parameterized markov chains but I never seen anyone describe it this way, is it just not a good approximation/hiding too much of the technical workings to be true?

https://chat.openai.com/share/796f9635-04f5-4591-8a0f-a3b477...

Instead of writing an essay, just say concisely what you have in mind here

what is a mind? what does it mean to reason?

Define “mind”