Cool project, but it reminds me of one of the golden rules of AI “if you can solve it with a simple rules based system, you should”
Having to tell chatGPT to make sure it gets it’s math right is not confidence inspiring.
Having to tell chatGPT to make sure it gets it’s math right is not confidence inspiring.
Wonderful example @mintplant.
Two sub-questions: 1) I thought LLMs worked by predicting the most likely next token. Why would telling it to check it's math actually make it's predictions more accurate?
2) Why would asking it to check it's math change the results it produces? Shouldn't it just do the same thing a second time and produce answers that are just as (in)accurate as it did the first time?
Saying it "split it evenly" and giving it numbers is something like saying "find me the hill next to this valley". This will lead you to the valley on the left, then to the tiny hill right behind it. The hill right behind it in this example is where all the weights about splitting stuff evenly are hanging around.
Giving it "split evenly and make sure the math is correct" is like saying "find me the the hill next to this valley, but make sure it's the tallest". So it will lead you to the same valley, but then to a different hill, because of adding "tallest" it will lead you to the intersection of where all the weights about splitting stuff evenly and correct math were found.
I.e. there is other stuff you can engineer about - saying stuff like "Write the result step by step and check for correctness." afterwards will navigate it to the same hill but then towards the "all math tests are here" hill which is where step by step calculations weights are and it will find it's new home there, giving you a more detailed output and more chances to be right since the prediction is easier - basically it "splits" the task into smaller, easier to predict chunks, like you split ugly code into small testable functions to understand it.
Then I told it to sound even more human and it does even better.
But it is not deterministic. For each token it picks at random, from a probability distribution generated by the neural network. You can make it deterministic if you set temperature=0. In general it is stochastic, if you sample again with the same prompt you get a different answer.
We don't compute in our heads, we use calculators. chatGPT could have a calculator too. Why kill a fly with a sledge hammer?
In some papers they even use named symbols. So you could have "[code]width=10; height=20[/code]. What is the area? ... The area is [code]width * height[/code].