Hacker News Reader

Comment by photon_lines | Hacker News Reader

There's actually a pretty simple solution to this that I thought about testing out and it involves asking the model to re-construct the problem using a logic language (like Prolog) and asking it to execute this type of program in order to come up with a solution rather than attempting simple chain-of-reason training / other methodologies of getting the model to 'reason' through some of these examples. People forget that humans don't come up with their logical models out of the blue - it takes years of elementary school in order for us to understand the world and solve problems in it. The logic programming approach I'd say is really promising but you would need to feed the LLM a LOT of examples in order for it to work, and currently I'm not even sure that we have enough training data in order to implement something like this.

View on HN

> asking it to execute this type of program in order to come up with a solution

I may be showing my ignorance about this tech here, but I believe the LLM doesn't even try to solve a problem; they try to generate a discourse that could pass as a solution or answer to the problem; that's more or less what the abstract states if I understand it correctly. But in no way does it try to apply some sort of mechanical reasoning like inference engines do.

To me the solution to this is to associate LLM with mechanical computations, that is an inference engine or an equation solver, rather than recombining the millions of solutions for similar problems it has seen in its training set. I believe I remember reading about teams attempting this approach. I can imagine for instance that if the LLM is in some way able to ask questions and use the answer, maybe it could just generate a prompt for an equation solver and include the result in its answmer.

Given that Alice has 13 brothers and 31 sisters, we can update the Prolog program with this information. We need to adjust the fact about Alice's siblings and then use the rule to calculate the number of sisters her brothers have.

Here is the updated program:

```prolog % Facts siblings(alice, 13, 31). % Alice has 13 brothers and 31 sisters

% Rules % Alice's brothers have M + 1 sisters, since Alice is also their sister. sisters_of_brother(TotalSisters) :- siblings(alice, _, M), TotalSisters is M + 1.

% Initialization goal :- initialization(main).

% Predicate to find and print the total number of sisters Alice's brothers have main :- sisters_of_brother(TotalSisters), writeln(TotalSisters). ```

In this program, the fact `siblings(alice, 13, 31)` represents that Alice has 13 brothers and 31 sisters. The rule `sisters_of_brother(TotalSisters)` calculates the total number of sisters that Alice's brothers have by adding 1 to the number of sisters Alice has (since Alice is also their sister).

When we run this program, it will print the total number of sisters Alice's brothers have:

1. Alice has 31 sisters. 2. Each of Alice's brothers will also consider Alice as a sister. 3. Therefore, each brother has 31 + 1 = 32 sisters.

The output of the program will be:

``` 32 ```

Thus, Alice's brothers have 32 sisters.

Problematic in the real world meat space to say the least.

Assuming Alice indeed has that many brothers and sisters (possible) it's highly unlikely they all have the same pair of unique parents.

The Set {} of siblings that share at least one parent with Alice (the siblings of Alice) can easily include one brother of Alice (same father, different mother) who doesn't share a common parent with any other sibling of Alice.

I honestly thought about this recently when I was trying to see the limits of Claude Opus. Some of the problems I gave it, what if instead of telling it to solve the problem I asked it to write the script and then give me the command and inputs needed to properly run it to get the answer I needed. That way instead of relying on the LLM to do properly analysis of the numbers it just needs to understand enough to write the logic.

It is an interesting prospect but I feel like it has some limitations. For math problems like this one, yeah it should be simple to write a script to do it. But it does first have to understand the core thing here that Alice would be one of the sisters of the brother to write the script accordingly.

But I would think this would not scale well when dealing with far more complex issues, particularly ones that may not just be simple math logic. If the request was to write a scientific paper for example, it might have to make several scripts and call them multiple times, and that assumes that it understands what it is doing enough to properly make the script.

I don’t understand why LLM’s aren’t already set up to do what you describe automatically behind the curtain. Extract a math equation from text (LLMs are good at translating between languages right?) and immediately evaluate it on the host CPU.

LLM is the equivalent of recalling your times tables. Computer arithmetic is the equivalent of re-computing your times tables.

> I don’t understand why LLM’s aren’t already set up to do what you describe automatically behind the curtain.

LLM-based systems with tool use (which this is an application of) often are, to an extent, the issue is tuning the (behind the scenes, system) prompting so that they use appropriate tools in every case where they should, and do so correctly. (There's also a cost factor involved since behind-the-scenes tool use means multiple LLM round trips to answer the question, so tuning the system to use tools more aggressively makes the system more expensive.)

ChatGPT does do this sort of process for arithmetic now; it converts wordbased problems to mathematical notation and then solves.

I'm not sure what you mean by 'it will not scale well.' When we humans learn that we make a mistake - we make a note and we hold the correct answer in memory - the next time we're prompted with a similar prompt, we can use our old memories to come up with the correct solution. I just did a simple test for this same exact problem using ChatGPT 3.5:

'Can you reformulate the following problem using Prolog? When you execute the program - what answer do you get? 'Alice has 8 brothers and she also has 5 sisters. How many sisters does Alice’s brother have?'

Even though it came up with a proper Prolog formulation - it failed to account that Alice is one of the sisters - so I gave it a simple prompt for this and it was able to correct the formulation and come up with the proper solution:

My prompt after ChatGPT gave me an answer of 5: 'Yes but you aren't including Alice in your logic. Shouldn't the correct answer be 6 (since Alice is also a sister)?'

Response:

'You are correct. The correct answer should account for Alice being one of the sisters. Let's adjust the Prolog program to include Alice in the calculation:

% Facts brothers(alice, 8). sisters(alice, 5).

% Rules sisters(Brother, NumSisters) :- sisters(alice, Num), NumSisters is Num + 1. % Include Alice herself

% Query ?- sisters(alices_brother, NumSisters).

In this updated Prolog program:

The rule for determining the number of sisters includes Alice herself by adding 1 to the number of sisters Alice has.

When we execute the query ?- sisters(alices_brother, NumSisters). in Prolog, we should get:

NumSisters = 6.

This correctly reflects that each of Alice's brothers has 6 sisters, including Alice herself.

When I prompt it to answer this question using Prolog now, I always get the correct answers no matter which integers I feed in, so it seems like the model is able to generalize after I show it where it's 'reasoning' was wrong. Basically, my suggestion is to get the model to recognize logic puzzles like this and try to formulate them in terms of logic programming queries which it can use and tune in order to come up with correct answers rather than simple auto-associative chain of reason training which current GPT models rely on, but like I said - this is my hypothesis and I believe this would work much better in getting these models to 'generalize' than the current approaches we're using. Hopefully this helps.

"My prompt after ChatGPT gave me an answer of 5: 'Yes but you aren't including Alice in your logic. Shouldn't the correct answer be 6 (since Alice is also a sister)?'"

Useful, if you know what the answer is. What happens if you don't give it the correct answer?

What happens when you don't give a human being the correct answer? :)

What if you prompt it "You seem to have accidentally included Alice. The correct answer should be 4"?

Yup this is a good example. This is because the model has no conception of what 'causality' is or how to try to come up with a correct 'model' - humans have a visual system which helps them out, but for LLMs I can definitely see your point and yup in these instances - if you feed in garbage data then yeah - you will get garbage out.

When you ask again the prompt includes the context of your previous question and correction. When I ask the prompt doesn't have that context so the model fails to give me the correct answer.

I'm using the default free model in the app, based on GPT4.

Yup - well this is where my suggestion is to change the GPT architecture. You can think of having a logic program function as the 'frontal lobe' of the general pre-trained auto-associative model. This 'frontal lobe' region would try to come up with logical sequences to go along with it's internal auto-associative representations. Of course - the logic programming piece is just one approach - maybe chain of though or chain of reason prompting could work here too as many humans I think use this chain-of-reasoning approach themselves. Logic programming to me would function as a suggested shortcut.

I took one of the problems that gpt4o got wrong, and asked gpt4o what tools it could use. It suggested and wrote prolog for me that (with one pass back to get things defined in the right order) which correctly worked out the answer.

I just tried that with ChatGPT 3.5 (4o stopped responding after I asked the initial question and it produced the wrong answer). Here's the Prolog it generated:

    % Define the number of brothers and sisters
    brothers(4).
    sisters(1).

    % Predicate to calculate the number of sisters a brother has
    brother_sisters(NumberOfSisters) :-
        sisters(NumberOfSisters).

    % Query to find out how many sisters a brother has
    ?- brother_sisters(Sisters).

Right, and do you verify the result?

You have to know what the answer is supposed to be before you can write a test case.

Yup - well you feed in the prompt along with an answer and you get the model to produce outputs and check for discrepancies. If the answer is wrong then the model adjusts -- this is the way backpropagation works....I think there are huge advantages in using logic languages in order to represent some of these data sets rather than simple English or the current chain-of-thought reasoning approaches -- backpropagation as an example isn't really used in the human brain, but it leads to great results in mimicking how neural networks 'learn' - in the same way, we don't have to have the full formal picture of how humans model the logical world in order to achieve great results. We can simulate this using logic programming or even general programming or at least that's my conjecture.

I can have more confidence that my calculations are correct using a calculator compared to doing it by hand, even if I don't know the exact right answer beforehand.

I’m curious how this would work considering knowledge can be fuzzy.

Like if I’m out camping and I sit on a log or a rock those things are not what people usually think of as chairs but they can serve as chairs in that situation.

You can get models to actually show that 'logs' could function as 'chairs.' You're forgetting that we humans also learn this as well, but we learn this in a much simpler manner than LLMs though so someone has to explicitly let models know what assumptions they can make. You get the LLM to write Prolog programs and learn associations in this manner. As the model gets better at logically modelling the problems - the solutions to prompted problems like this should get better.

If that kind of thing worked, we'd have been doing it long before LLM chatbots.

Yet tools like GPT4o can do this.

It's not a trivial problem, taking a human written description and rewriting it as a prolog program.

Reply on news.ycombinator.com