Hacker News Reader

Comment by londons_explore | Hacker News Reader

The training data doesn't seem to be great quality...

        "instruction": "Calculate the sum of each column in the following table.",
        "input": "2   |  3  |  4\n   5   |  6  |  7",
        "output": "2+3+4 = 9\n5+6+7 = 18"

I think better results could be obtained by manually going through these 52,000 training examples - even a couple of seconds per example should be enough to weed out obviously wrong ones, and should only take <$1000 of human time.

View on HN

Don’t even need human time. Just run it through davinci. Is this correct, yes or no? Or MTurk. Can cost less than $500.

Not sure about this specific example, but the prompts used for training were generated by davinci. The researchers only used 175 human-written prompts, and these were fed into davinci to generate the other ~52k used for fine tuning LLaMA to make it follow instructions. They had humans review the generated prompts to eliminate bad ones before fine tuning, but humans mess up.

I wonder how davinci would respond if every single output was fed back into itself with a prompt like "does this contain any errors?". That probably wouldn't work very well.

Funny enough it would.

LLMs have proven to be quite good at evaluating their own output. Think of things like "Think step by step" and how that improves output.

A trick I've started to use with davinci is prompting it to use a calculator for all math

I preprompt it to believe it's math cannot be trusted (because it can't) and to instead output "!c <insert basic calculations>" any time it would otherwise try to solve something

The next step I'm working on this weekend is piping that to Wolfram Alpha and seeing if I can put together a wrapper to make it "seamless" in terms of responses

Well, MTurk is selling human time, so...

One of the famous use cases of InstructGPT is being able to infer things from structured data, even unusual questions.

Yes... but the answer is wrong...

If you train it on wrong answers, it will generate more wrong answers...

The correct answer should be:

    2+5=7\n3+6=9\n4+7=11

Training a model to do math is impossible. If you tell a child that 1+1=2 10+10=20 100+100=200 an "AI" can not figure out that 1000+1000=2000 like a child can.

>Training a model to do math is impossible.

https://www.deepmind.com/blog/discovering-novel-algorithms-w...

A language model cannot, by itself, figure that out, at least not to arbitrary complex inputs.

But it’s a fairly sweeping statement to say that AI cannot do it.

Of course it is possible for LLM model to learn this!

The problem was with token embeddings, not with math.

Has this actually been tried? It would be trivial to generate a very large dataset of simple additions and see if an LLM gets it. Maybe it wouldn't serve an immediate practical purpose but I'm curious.

I did try this, although I'm not an AI expert. It will work great for everything that are in the training set, but if there is a hole the model will have trouble. Lets say we train a model with the numbers 1 and 2. 1+1=2, 1+2=3, 2+2=4 we now give the model 2+1 it will likely guess 2 or 4 because those are the ones that have 2 or 1 in the correct place. A neural network can't do logic. It's good at other things though, for example parsing, a language model could parse human input "two plus one" to 2+1 and then just calculate the answer using a math library.

I wish I still believed this.

Reply on news.ycombinator.com