Why would it be expected for LLMs to be able do logical deduction?
What’s actually happening under the hood is so ass backward bizarre people jump to incorrect conclusions. It’s amazing what you can do with that much data and computational power.
Well, yes, narratives that look like reasoning and have accurate cobclusions are more common in their training data with langauge that references reasoning before them, so prompts that call for reasoning explicitly produce narration that looks like reasoning. (And which has more accurate conclusions, too.)
Something that came up often in my application was wanting to express some orbital elements as vectors rather than scalars, to be able to do some things more concisely (and avoiding having slow trigonometric functions strewn all over my code)
For trying to figure these out, generally GPT-4 performance was poor on accuracy and I was using it just as a tool to look up terms I would plug into google to find a better source that won't confidently lie to me. However sometimes it was doing an admirable job of transforming mathematical terms. This often can be done with just knowledge (like knowing sqrt(2)^2 = 2) and applying transformations on tokens - which is very much in the scope of these AIs. Technically that isn't much logical deduction yet, but there was a few cases where it impressed by stating things like "Since we know vectors K and V are perpendicular, we can ...".
Certainly looked like the beginnings of some basic logical deduction sometimes - even if getting halfway correct answers required me to restart prompts a dozen times each.
As a side note, GPT-4 (not sure about other models) is capable of doing logical deductions when prompted.
But not capable of doing logical deduction of the data it is trained on, just the data you give it.
It is very stupid about all the data in its training corpus, since it is encoded like a grammar and not a knowledge base.
This wouldn't be a problem if not for there being a million times more data in the trained set than what you can give it. But this mean that we can't really train the AI to be smart about a wide range of things, since the training corpus is stupid and the live data is very limited. (And it isn't even that smart about the live data, given how expensive it is to compute for that little amount of knowledge)
The expectation has been set to human-level understanding and explainability by everyday common people who don't need to know about how it works. Given it lacks explaining basic logical deduction and even regurgitating its own mistakes, we can't even begin to compare this to humans as it is not the same.
We're just searching for the prompt that can reliably break these LLMs to show their lack of reasoning and at some point eventually someone is going to find it and it will break all of them.
What gives you the idea that there's some universal prompt to "break" LLMs? What does a "broken" LLM even look like to you? Do you just mean that it gives back a wrong answer? It's the only interpretation I can come up with, and they already do that all the time.