Comment by LightMachine | Hacker News Reader

Comment by LightMachine | Hacker News Reader

Parent Full thread

If we want to test these beasts in logic, we should probably start using actual formalized logic, rather than English. In just one test, Gemini flopped hard, while GPT-4-Turbo nailed it. Here is my prompt:

    Below is a well-typed CoC function:

    foo
    : ∀(P: Nat -> *)
      ∀(s: ∀{n} -> ∀(x: (P n)) -> (P (n + 1)))
      ∀(z: (P 0))
      (P 3)
    = λP λs λz
      (s (s (s z)))

    Below is an incomplete CoC function:

    foo
    : ∀(P: Nat -> *)
      ∀(f: ∀{n} -> ∀(x: (P n)) -> (P (n * 3)))
      ∀(g: ∀{n} -> ∀(x: (P n)) -> (P (n * 2)))
      ∀(h: ∀{n} -> ∀(x: (P n)) -> (P (n + 5)))
      ∀(z: (P 1))
      (P 17)
    = λP λf λg λh λz
      {{FILL_HERE}}

    Complete it with the correct replacement for {{FILL_HERE}}.
    Your answer must contain only the correct answer, and nothing else.

- *GPT-4-Turbo answer:* `(f (g (h (g z))))` (correct)

- *Gemini Advanced answer:* `h (h (g (f z)))` (wrong)

Also, Gemini couldn't follow the "answer only with the solution" instruction and provided a bunch of hallucinated justifications. I think we have a winner... (screenshots: https://imgur.com/a/GotG0yF)

> If we want to test these beasts in logic, we should probably start using actual formalized logic, rather than English.

Why? Do you use formalized logic when discussing with other people about topics that involve logic? You know, a logic riddle or a philosophical question can be understood and processed even if the only tool you have is your native language. Formalized logic is a big prerequisite that basically cuts out the vast majority of Earth population (just like coding). Now, if you mean that in BENCHMARKS they should use formalized logic syntax, probably yes. But in addition to plain language tests.

Because once an AI becomes proficient at formalized logic, it:

1. Completely stops hallucinating, since we can demand it to internally prove its claims before showing the answer;

2. Stops outputting incorrect code (for the same reason);

3. Starts being capable of outputting complete projects (since it will now be able to compose pieces into a larger code);

4. This is also what is needed for an AI to start self-improving (as it will now be able to construct better architectures, in a loop).

That's why I argue getting the AI competent in logical reasoning is the most important priority, and we'll have no AGI until it does. After all, humans are perfectly capable of learning how to use a proof assistant.

Moreover, if an AI can't learn it no matter how hard it tries, you can argue that there is at least one human capability that the AI can't replicate, thus it isn't an AGI.

Humans mostly don't use logic, so how are you defining "AGI"? ChatGPT + plugins is pretty close to how humans think ("biased random word-association guess + structured tool")

AGI implies there are no cognitive tasks that some humans can perform, yet that this AI can not perform. Otherwise, what is the point?

ERRATA: I just noticed GPT-4 mixed up `h` and `f`, so it also got it wrong. This is a draw. Both AIs (and, apparently, myself) are terrible at reasoning. Guess we're not curing cancer with computers anytime soon :')

Kudos for the correction, but you should really put this, by far the most important context, in your original post.

I would love to, if YCombinator allowed me. The "edit" button is missing. I've edited on Reddit and other places where I posted this test.

I've been using GPT-4 to help me understand my MSc mathematics course and I've noticed this sort of stuff more and more as I start to look at the answers, always confidently written, in detail.

Way back when GPT was just fresh on the scene I had terrible anxiety about "what is the point of my whole career or even learning any more" but these days I'm much less concerned. I'll ask it something relatively simple, like "make a sentence out of words 'a', 'b', & 'c'" for it to reply with "'a' 'b' 'd' 'e'" for me to then correct it with "oh, you didn't use c" for it to then respond "sorry, here - 'a', 'c', 'd', 'f'" etc.

Definitely an amazing complimentary tool but when they say "can make mistakes, check important..." that's essential.

This sort of issue holds with all kinds of prompts, on both platforms. I most recently (to test Bard's image generation capabilities) was asking Bard/Gemini to generate home designs using highly specific prompts -- layout of the house, materials for the facade, window placement and style, etc -- and it was shocking how frequently it would just ignore critical pieces of the prompt, and then continue to ignore when corrected.

Such a weird test. 99.9% of humans wouldn't even understand the question, let alone be able to formulate a coherent answer for it.

Being able to answer these questions is a pre-requisite for AGI. After all, there ARE humans capable of doing that, so, if the AI can't do it no matter how hard it tries, then that means there ARE human capabilities that the AI can't replicate (thus, it isn't an AGI). And it seems like no LLM is making any progress at all in that kind of prompt, which is why I use it as a core benchmark on my "AGI-meter".

Though humans aren't able to do it in a chat session. Being able to work on the problem in the background for a couple days may be a prerequisite for AI to solve these problems. And such would require money from the asker.

Anyone familiar with the syntax / jargon should be able to answer this specific problem in ~5 seconds of thinking, though. And I mean it, even a 10yo kid should...

I think you'll be using that meter for a long time, then. I don't really know anyone who's under the impression that the current direction of LLMs are going to produce AGI, it seems as if you're barking up a tree most people aren't really concerned exists.

That's fair enough

Except there’s a lot of not-so-informed people who think AGI was always here when chatgpt came out. Even more that think it’ll get there very shortly based on just bigger and bigger LLMs. Many have argued as such here on HN.

You're making a completely incoherent argument -- that if it can't do a single task that some percentage of people can do, that it's not at intelligent, when there is nobody on earth that can do everything that some small percentage of people on earth can do, by definition.

Why is this relevant to the performance of a computer program? It makes sense to me that computer programs & humans should continue to be judged by different standards.

If a good chunk of humans can't pass your "general intelligence test" then it's not by definition a general intelligence test unless humans are not generally intelligent.

which is better than formulating a coherent but wrong answer

Gemini destroyed by facts and logic.

Is that supposed to be a P((n+3)) in the type of "f" for the second case or am I misunderstanding this hard?

No, it is `n * 3`. The challenge is simple: starting from "1", we must reach "17" by combining the operations `x * 3`, `x * 2` and `x + 5`. What is embarrassing is that Gemini manages to read the formal jargon and understand the challenge just fine. Yet it fails to combine these operations to get from "1" to "17", which is something most 10 yo kids would be able to do.

Yeah, seems like I got that right. That graduate-level course must have worked. But then:

    (f (g (h (g z))))

results in:

    ((((1 * 2) + 5) * 2) * 3) = ... not 17?

while it would work if the type of f was corrected.

Or, again, am I missing something?

You are right and I had the same reaction. The correct answer should have been: (f (h (g (h z)))) AKA ((1 + 5) * 2) + 5.

Is it not ironic that the supposedly test of AGI is flawed and its human designer fail to see it and denied it when presented with facts. Maybe the test designer is hallucinating just as much as those LLM? :)

It's pretty amusing and it is not the first time I've seen this. Random example: https://news.ycombinator.com/item?id=38387168

It's a little scary that it can be so hard to evaluate the correctness of these LLMs even when we are paying close attention and looking for mistakes. Or maybe the scary part is that we can become biased when we want to believe.

Oh fuck. Well, in my defense, nobody is claiming I'll design fusion reactors and cure cancer by 2027

No, but we at least acknowledge you a general intelligence like all humans. I'm not sure when agi - artificial general intelligence began to mean anything other than artificial and generally intelligent.

agi may as well be God, the bars some people have.

> The correct answer should have been: (f (h (g (h z)))) AKA ((1 + 5) * 2) + 5.

Isn't that (h(g(h z)))?

And, FWIW, at least in my test, Gemini gets that in its final answer, though it failed in the two other drafts:

https://g.co/gemini/share/c922e7ef62aa

ChatGPT sputters:

https://chat.openai.com/share/25abbf47-2ed4-4635-a351-90a9a6...

(ETA more Gemini testing suggests its correct answer was a one-off)

Oh, lol, you're right. Seems like I'm dumber than both AIs. GPT-4 mixed up `h` and `f`, so it also got it wrong, so this is a draw and both AIs (and, apparently, myself) are terrible at reasoning. Guess we're not curing cancer with computers anytime soon :')

so much for "Anyone familiar with the syntax / jargon should be able to answer this specific problem in ~5 seconds of thinking, though. And I mean it, even a 10yo kid should..."

This isn't representative of real-world usage since you don't let it think.

> I think we have a winner...

It makes me sad that the complete and total lack of an objective way to measure these products means that the coming decades will be filled with this kind of hyper-specific gotcha test made in inappropriately confident internet posts.

Literally this could have been down to one extra book in someone's training corpus, or a tokenizer that failed to understand λ as a non-letter. But no matter, "we have a winner!". It's the computer science equivalent of declaring global warming a fraud because it snowed last night.

Disagree. People are going to rely on these things, and when they make stupid but confident mistakes (i.e. they produce bullshit), they are dangerous.

An AI system that produces right answers 90% of the time but 10% of the time drives your car into a lane divider, or says "there are 4 US states that start with 'K'" or "Napoleon was defeated at the Battle of Gettysburg" is worse than useless: It's dangerous.

As long as we call it a bullshit parlor trick, no problem. But unfortunately people are making important decisions based on these things.

You're completely wrong. Gemini can perfectly understand what is being asked, so this isn't a syntax issue. Notice that, on the answer, it even states the solution: "starting from 1, and combining `* 2`, `* 3` and `+ 5`, we must reach 17`". So it does fine with reading the formal syntax, yet it fails to combine these operations to get from "1" to "17", which is something most 10 yo kids would have no trouble doing. And that's after millions spent in training. Now tell me again this is the architecture that will figure out the cure of cancer?

Reply on news.ycombinator.com