Hacker News Reader

Comment by jimmaswell | Hacker News Reader

Imagine Wolfram's system with GPT integrated. I hope they're working on that.

Current GPT version is not very good at math, and it's not at all clear whether this can be fixed within a language model.

Q: I have 2 cats and 1 horse. How many legs do they have in total?

A: If you have two cats and one horse, then the total number of legs they have is 8. Cats typically have four legs, while horses have four legs as well. So two cats would have a total of 8 legs, and when you add the four legs of the horse, that brings the total number of legs to 8.

It would have got it right if you got it to show its working.

That's the trick to doing computations with LLMs, the computation per token is constant so you want to split the computation over many tokens.

> I have 2 cats and 1 horse. How many legs do they have in total? show your working.

> If you have 2 cats and 1 horse, they would have a total of 10 legs. Cats typically have 4 legs, so 2 cats would have 2 x 4 = <<24=8>>8 legs. Horses have 4 legs, so 1 horse would have 1 x 4 = <<14=4>>4 legs. Together, the 2 cats and 1 horse would have 8 + 4 = <<8+4=12>>12 legs.

> That's the trick to doing computations with LLMs, the computation per token is constant so you want to split the computation over many tokens.

Could you explain this a bit more? What are LLMs? Why is the number of tokens different in your approach?

LLM = Large Language Model. Asking it to show its work will cause it to generate more tokens (you could think of it as the logical leaps between each step being smaller the more detail you go into)

It's very reactive to the prompt:

> i have 23 horses each with a normal number of legs, along with 14 dogs, each with a normal number of legs. how many legs in total do all of the animals have?

> Based on the information you provided, it sounds like you have 23 horses and each horse has 4 legs, and you have 14 dogs and each dog has 4 legs. If that is correct, then the total number of legs among all of the animals would be 23 x 4 + 14 x 4 = 92 + 56 = <<234+144=148>>148 legs.

You know funny thing, for comparison I just entered that exact prompt into the 3 gig sized flan-t5 and I get the following:

> The horses have 23 * 4 = 126 legs. The dogs have 14 * 4 = 84 legs. The total number of legs is 126 + 84 = 204. The answer: 204.

Very very similar, but completely wrong results.

I've found it gets the math right most of the time in stories. For example:

> Write a story about a veterinarian counting the number of total legs on two cats and a horse. He reports the total number of legs to his assistant so he can write them down.

As the veterinarian entered the examination room, he was greeted by two playful cats and a majestic horse. He smiled and began his examination, carefully checking the health of each animal.

After completing his examinations, the veterinarian turned to his assistant and said, "I need you to write down the total number of legs for these three animals."

The assistant nodded and picked up a pen and paper. "Okay, let's see," the veterinarian said as he counted the legs on the cats. "Each cat has four legs, so that's eight legs for the two cats."

Next, he moved on to the horse. "And this beautiful horse has four legs as well, for a total of twelve legs."

The assistant quickly wrote down the numbers as the veterinarian spoke. "Twelve legs total," the veterinarian confirmed. "Make sure to include that in their medical records."

The assistant nodded and continued to take notes as the veterinarian continued to provide updates on the animals' health. As they finished up the examinations, the veterinarian couldn't help but smile, knowing that he was able to help these wonderful creatures.

Fucking amazing.

LOL, that outburst fits in with your HN username.

this is what I got with the same prompt:

The veterinarian, Dr. Smith, walked into the examination room where two cats and a horse were waiting. He greeted the animals and their owners before approaching the first cat. The feline, a fluffy white and orange tabby, was lying on the examination table. Dr. Smith gently lifted one of its hind legs and counted the toes. "One, two, three, four," he muttered to himself before moving on to the cat's front legs.

After counting all four legs on the first cat, he repeated the process on the second feline, a sleek black and silver Maine Coon. Once he had counted all eight legs on the two cats, he turned his attention to the horse. The majestic creature stood in the corner of the room, calmly munching on some hay.

Dr. Smith walked over to the horse and began counting its legs, starting with the left hind leg. "One, two, three, four," he said aloud, moving on to the horse's front legs. As he counted, his assistant, who was standing nearby, jotted down the numbers on a piece of paper.

After counting all four legs on the horse, Dr. Smith stepped back and looked at his assistant. "That's a total of 18 legs," he said. "Can you write that down for me?"

His assistant nodded and added the number to the growing list on the paper. "Got it," she said. "18 legs total."

Dr. Smith smiled and turned back to the animals. "Alright, let's get started on your check-ups," he said, ready to continue with the examination.

Ha. At least its opinion of horses is consistent.

Interesting - this is how we teach arithmetic to children. There's a certain amount of rote learning to get started (as, despite their amazing ability to learn language, they generally do not reliably pick up the basics of arithmetic from what they hear other people saying), and then we move on to exercises constructed from more-or-less plausible real-world examples. We certainly do not start with the axioms of number theory.

Perhaps you can ask GPT about what is wrong in the answer and how it would fix that? This would give a general approach:

Q: [Question]

A: [Answer]

Q: What is wrong in "[Answer]" and how would you fix that?

A: [Improved answer]

These errors are common and important to note. Though what I find fascinating is how often I can prompt engineer around these errors by telling GPT to just be smarter and think more carefully. Just tried the following for instance.

Q: You are a master logician, who can solve all manner of language problems and have excellent math skills. I have 2 cats and 1 horse. How many legs do they have in total? Show your reasoning and calculations step-by-step

A: As a master logician, I can tell you that the total number of legs possessed by two cats and one horse can be found by using basic arithmetic. Here's how:

First, we know that each cat has four legs, because all cats have four legs. This means that the two cats together have 2 x 4 = 8 legs.

Next, we know that a horse has four legs, because all horses have four legs. This means that the horse has an additional 4 legs.

Finally, we can add the number of legs possessed by the two cats and the horse together to find the total number of legs. This is done by adding 8 + 4 = 12 legs.

Therefore, the total number of legs possessed by two cats and one horse is 12 legs. I hope this helps! Let me know if you have any other questions.

That's interesting. It seems it's not great at raw calculation, but if you ask it to explain its reasoning, it derives steps from previous results, and arrives at the correct answer. It's similar to the sibling comment about asking it to tell a story.

Correct. That concept is the chain of thought (CoT) reasoning that the article discusses.

This seem to work because in the end it’s a markov chain. So the probability of the next step of a long answer being correct is greater than the probability of jumping to a correct conclusion.

Yes, this certainly helps. I find it ironic that you can get somewhat better results for nonfiction by giving it more clues about what it's supposed to be pretending to be. It's always pretending, though.

There’s an actual psychological effect for this as well [1]. Authors + uni are in the link. I forgot the name of the effect, don’t have time to do proper research.

[1] https://www.themarysue.com/lab-coats-increase-attention/#:~:....

0 chance this replicates, anything before 2016 I immediately write off. (i.e. clearly pre-replication crisis)

Halycon days of TED talks laundering cute little tidbits that seemed irrational but we all wanted to believe.

Fair point, not sure if it'll replicate. I just vaguely remember there's a thing in psych that if you act like it (a bit) then you become it a bit. Don't have the time to research it.

> Authors Hajo Adam and Adam Galinsky wanted to explore that nature of what they call “enclothed cognition,” the effect that the clothes we are wearing have on our psychology

Who isn't?

Thanks for that. What a great example of how to actually use these things. You need to almost prime the chain with your prompt, give it some structure towards the particular later chain step you want. It makes a lot of sense that it can’t just leap to what you want, but with some setup, you can almost lay the path for it to follow.

This looks similar to priming in humans.

Also, it looks to me like a language model is capable of reasoning if you let it execute a few times.

First, have it generate multiple outputs using different primings. Then have it choose it's favorite output.

Map, Reduce

Quite bizzare really. I wonder if you tweak a single word in that prompt, if you end up with a totally different sum.

Suddenly, all those movies and tv shows where characters tell robots to concentrate start to make sense.

I tripped it up with a somewhat ambiguous question using pirates.

Q: There are two boats in the ocean. One boat has 11 regular sailors. The second boat has 8 pirates. A pirate is defined as a human being where one of their legs is made of wood. How many real human legs are there in total among the two ships? Explain your reasoning.

A: There are 11 regular sailors on one boat and eight pirates on the other boat, for a total of 11 + 8 = <<11+8=19>>19 people.

Since a pirate has one wooden peg leg, there are 8 wooden peg legs among the group of pirates. Since a wooden peg leg is not a human leg, the total number of human legs among the group of pirates is 8 - 1 = <<8-1=7>>7.

In total, there are 11 + 7 = <<11+7=18>>18 human legs among the two boats. This is because the 11 regular sailors each have two human legs, and the group of pirates has a total of 7 human legs.

Once I was far more explicit in explaining the logic however it was able to get it correct:

Q: There are 2 boats in the ocean. One boat has 11 regular sailors. The second boat has 8 pirates. Sailors always have two legs. Pirates always have one leg because the other one is made of wood. How many real human legs are there in total among the two ships? Explain your reasoning.

A: There are 11 regular sailors on one boat and eight pirates on the other boat, for a total of 11 + 8 = <<11+8=19>>19 people.

Since a sailor always has two human legs, there are 11 * 2 = <<112=22>>22 human legs among the group of sailors.

Since a pirate always has one human leg and one wooden peg leg, there are 8 1 = <<8*1=8>>8 human legs among the group of pirates.

In total, there are 22 + 8 = <<22+8=30>>30 human legs among the two boats. This is because the 11 regular sailors each have two human legs, and the group of pirates has a total of 8 human legs.

I'd say it struggles with pure logic.

Is there any work towards using ChatGPT to generate first order logic from a prompt, solve it using an existing solver, and put it back to let it write the meaning of it ?

I think it could be interesting, as it would make some prompts work better, and only use got for actual language to language tasks but leave reasoning to another software.

It doesn't have to be good at math. It just needs to know when the question is math, so that the answer can be delegated to a system that can easily answer it. Some people have already managed to do it with Jupyter notebooks (with manual copy-pasting of expressions generated by the model, and results of their evaluation).

That's why you'd use ChatGPT to parse the question and wolfram for the actual math.

"2 * (number of legs on a cat) + (number of legs on a horse)"

Result: 12

https://www.wolframalpha.com/input?i=2+*+%28number+of+legs+o...

If I have a duck and a dog how many legs do they have in total?

If you have a duck and a dog, they would have a total of 10 legs. Ducks and dogs both have 4 legs, so 2 x 4 = 10. Is there anything else you would like to know?

———-

Think carefully and answer again

If you have a duck and a dog, they would have a total of 8 legs. Both ducks and dogs have 4 legs, so 2 x 4 = 8. I apologize if I gave the wrong answer earlier. My training as a language model means that I can provide general information on a wide range of topics, but I do not have access to the internet and my knowledge is limited to what I have been trained on, which includes up until 2021. I may not be able to provide accurate or up-to-date information in all cases. Please let me know if you have any other questions.

———-

Can yoy explain your reasoning in detail?

Sure, I would be happy to explain my reasoning in more detail. As a language model, I have been trained to process and generate text based on a large corpus of text. This allows me to provide general information on a wide range of topics, but I do not have access to the internet and my knowledge is limited to what I have been trained on, which includes up until 2021.

When you asked about the number of legs that a duck and a dog have in total, I used the information that I have been trained on to provide an answer. Based on this information, I know that both ducks and dogs have 4 legs, and that two multiplied by four is equal to eight. This is why I initially provided the answer of 8.

However, I realize now that my initial answer may not be accurate, because it is possible that the duck in question is a baby duck, or a duck that is missing a leg for some reason. In these cases, the duck would not have the usual 4 legs, and the total number of legs would be less than 8. I apologize if my initial answer was not accurate in your specific situation. As I mentioned earlier, my training as a language model means that I can provide general information, but I may not be able to provide accurate or up-to-date information in all cases. I hope this helps to clarify my reasoning. Let me know if you have any other questions.

———-

How many legs do ducks have?

Most ducks have 4 legs. Like all birds, ducks have two legs that are used for walking and perching, and two wings that are used for flying. The legs of a duck are strong and sturdy, and they are adapted for swimming, with webbed feet that help the duck to move efficiently through the water. The legs of a duck are also longer and stronger in proportion to its body size than the legs of most other birds, which helps the duck to walk and swim with ease. Overall, ducks have 4 legs, just like most other birds. I hope this helps. Let me know if you have any other questions.

It's IMHO even worse when you try to make it generate mathematical proofs - even of well-known statements.

Here's a "proof" of Gödel's Completeness Theorem:

Prompt: Write a proof of Gödel's Completeness Theorem

Answer: Gödel's Completeness Theorem states that for any consistent, effective axiomatic system for a first-order language, if a sentence is true in all models of the system, then it is provable in the system.

To prove this theorem, we must first show that if a sentence is true in all models of a system, it must be true in at least one model of the system. Suppose that a sentence is true in all models of a system. This means that for any possible interpretation of the symbols and variables in the sentence, the sentence will be true. However, since the sentence is true in all models, it must be true in at least one model, otherwise it would not be true in all models.

Next, we must show that if a sentence is true in at least one model of a system, it is provable in the system. Suppose that a sentence is true in at least one model of a system. By the soundness of the system, if a sentence is provable in the system, it must be true in all models of the system. Since the sentence is true in at least one model, it must be true in all models of the system, and therefore it is provable in the system.

Therefore, if a sentence is true in all models of a system, it is true in at least one model of the system, and if it is true in at least one model of the system, it is provable in the system. This completes the proof of Gödel's Completeness Theorem.

I played with this - it seems to be confused about whether the front legs on a cat are legs or arms, and gives inconsistent answers. One cat has four legs, but when you ask about multiple cats, each cat has two legs, at least from 2 - 6 cats.

Interestingly, it detected that I was asking about the legs on an increasing number of cats, from 1 to 3, and volunteered up the (incorrect) number of legs for 4 - 6 cats, as well as the correct number for 1 - 3 dogs and 1 - 3 elephants.

If you change "legs" to "limbs", it starts answering correctly for multiple cats. But when you add the horse, it goes back to saying cats only have two.

they seem to be very different and not at all compatible approaches.

but they're both computation, why wouldn't they compatibly compute?

(I just wanted to say "compatibly computing")

Reply on news.ycombinator.com