Your LLM Is a Capable Regressor When Given In-Context Examples | Hacker News Reader

Your LLM Is a Capable Regressor When Given In-Context Examples

arxiv.org

119 pointsTaurenHunter36 comments

The Friedman functions are sufficiently well known to be in the training set, likely in the form of some csv file from somebody testing other methods on them. Though likely any data set would not share the same x-values. Still, it's surprising performance when we're used to the LLM's getting confused by straightforward math problems and with arithmetic often crippled by the tokenization system.

They did invent their own functions to test if the results were due to these functions being on the training date. See the section on data contamination in the paper.

Agree it both kind of makes sense (regression is the best way to predict the next token in this context) and kind of ironic (LLMs can do high school regression but can’t do elementary school long digit arithmetic).

It's surprising, but in a weird way it makes sense if you think about it as "learning a pattern" instead of "doing math". The model doesn't have to understand logic as we know it (and this class of models doesn't seem to be able to do that in general), but it does have to be able to learn the pattern somehow, and we already know LLMs can learn patterns from the input context without any fine-tuning.

I'd like to try it on a dataset like Titanic. That might be a more interesting experiment.

Or to avoid training data bias, maybe a completely random dataset generated by some more-sophisticated means, like the scikit-learn data generator which can introduce clusters, irrelevant features, etc.

Initially I experimented with house prediction. But the models have probably seen this data, so I ended up experimenting with random functions.

I tried with linear regression with irrelevant features (e.g., NI 1/2 -> 1 informative variable, 2 total variables). The models still perform reasonably well.

All results can be seen in this heatmap: https://github.com/robertvacareanu/llm4regression/blob/main/...

Missing from the paper: "We were able to verify that none of the in-context exemplars were in OpenAI's training set."

I wonder if this paper will make it through peer review? Now that could be an interesting result.

The authors explicitly generated new random data to address this concern.

I'm a little skeptical. I grabbed a small dataset and asked ChatGPT to create a linear regression equation and then did the same in excel. Perhaps they estimate values for beta differently, maybe ChatGPT is secretly a Bayesian, but the output I got was different. It was close, which I will say is impressive. Also, maybe I missed it, but I think it would be valuable to add to the prompt to provide the estimates for the parameters. It seemed as if they sort of just took the prediction values, computed MAE, and took the estimate for what it said. As I type this, I do wonder if you could use it to compute estimates for the parameters iteratively, average them, and essentially compute a bootstrapped estimate for the population parameters..

I'm not surprised that a linear regression estimated by a method that knows it should be estimating a linear regression would beat everything else.

The traditional OLS estimator in Excel has all sorts of classical optimality properties when its assumptions (normality, linearity) are true, so no fancy neural net can outperform it even in principle (the only way to outperform it would be to have an informative prior for how you generated the data set's parameters). So if the LLM's beat, or even matched, Excel in that case, they would be thinking too narrowly.

Same with any other models where we know the form of the answer up to some unknown parameters.

If we want parameter estimates, that means we already have a functional form in mind. In that case, we get to use established statistical theory to design optimal estimators, whether by Bayesian or other methods. Black box neural magic wouldn't help (or it might help indirectly, in computationally intractable cases).

What we would want the LLM's to do, ideally, is explore the space of known/possible 'patterns' and perform well in situations where the underlying relationship exists, is not known in advance, and is known not to have a simple form we can describe. Much like they (and we!) produce text without being able to describe why they are producing that particular text, we would expect them to make those predictions without being able to explain them in terms of parameters and functions - not without a whole other layer of explainability machinery.

> In that case, we get to use established statistical theory to design optimal estimators, whether by Bayesian or other methods. Black box neural magic wouldn't help (or it might help indirectly, in computationally intractable cases).

Using 'established statistical theory to design optimal estimators' isn't trivial for most people. Black box magic might still be useful for them.

Let the LLM's magically write code for the optimal estimators, using existing theory; but that code will implement interpretable math with provable characteristics, as opposed to magic.

Maybe the LLM is just doing interpolation rather than regression?

I've played with math examples (a while back, not with recent models) where they make errors, but seem to get the magnitude right, so perhaps easy to find the closest points (or roughly closest) to interpolate between.

Interpolation is a legitimate technique for regression problems, a special case of the k-nearest-neighbor estimator (which is one of the methods they test against). Lots of related regression techniques involve a tradeoff between global trends and nearby points: KNN, kernel regressions, gaussian process regressions, generalized additive models, local regressions, mixed models/generalized linear models with some covariance structures - all of them more or less manifestations of the same underlying math. The tree based techniques - random forests, GBM and the like - they don't look like interpolations, but the more you zoom in to the leaf nodes of those trees, the more they look like averaging one or more local y-values.

Literal interpolation would be use a lot more often, except you can't practically do it in higher dimensions (even 2 isn't trivial).

Ive noticed the same on extremely small models aswell, magnitude is a positional encoding or a couple tokens, so its easy to grok?

Ok I tried to get this to work for a long time ~6 months ago. It doesn’t work. You can find datasets where it will work, but if you select random datasets you generally see 0 correlation between the predicted and actual values.

Hmmm... What do people think in cases like this. Is the author just straight up lying?

Well, the datasets used in the paper are all available (Appendix B) - so recreating the experiment seems possible.

What we are currently seeing in the comments are people trying random things and then saying “it doesn’t work”.

Chat examples with GPT-4: https://github.com/robertvacareanu/llm4regression/tree/main/... (the experiments used the API though)

For example, for Friedman #1, GPT-4 predicts 12.89 while the true value is 11.69 (https://chat.openai.com/share/177571ad-3845-46a1-952f-963647...)

For Original #1, GPT-4 predicts 83.63 while the true value is 80.39 (https://chat.openai.com/share/808da995-99e6-444a-94da-fc7cd5...)

Interesting - I tried completely cheating and got worse results!

https://chat.openai.com/share/6217cd86-2b0f-41b2-a36a-2558dd...

Interesting, thanks for sharing! I noticed when the LLMs were trying to explain the prediction they would sometimes erroneously generate that the relation is linear when it was not. This happened when I removed the following part of the prompt:

`The task is to provide your best estimate for "Output". Please provide that and only that, without any additional text.`

Examples of this are available in Appendix J.

Sidenote, but all the experiments were ran with the API. There are some differences between the Chat and the API, for example the Chat can generate and execute code. I shared Chats since they are easy to look at and to try.

If you have access to an API key, I made some google colabs:

Colab links:

- GPT-4 Example: https://colab.research.google.com/drive/1Bk9uBCBvzuX00Rex-t1...

- GPT-4 Small Eval: https://colab.research.google.com/drive/1_-uHvW2oLtcCXz0c-G_...

- Claude 3 Opus Example: https://colab.research.google.com/drive/105jUAGanp7ZLG-Q9Hei...

- Claude 3 Opus Small Eval: https://colab.research.google.com/drive/1-IH68TUuqf_CZyptSSr...

Correlation? What are you measuring, if I may ask?

The GitHub repo now contains (1) Links to Google Colab for examples + small-scale evaluation; (2) Jupyter Notebooks for small example + small-scale evaluation.

Colab links:

- GPT-4 Example: https://colab.research.google.com/drive/1Bk9uBCBvzuX00Rex-t1...

- GPT-4 Small Eval: https://colab.research.google.com/drive/1_-uHvW2oLtcCXz0c-G_...

- Claude 3 Opus Example: https://colab.research.google.com/drive/105jUAGanp7ZLG-Q9Hei...

- Claude 3 Opus Small Eval: https://colab.research.google.com/drive/1-IH68TUuqf_CZyptSSr...

‘We provide a total of 50 tuples… and then ask the model to generate y51, corresponding to x51…’

I wonder if testing a single point is masking a larger llm error.

The experiments were repeated with 100 random seeds.

Very interesting indeed. My current masters research is is doing related work and I can't get this to work on GPT4 (random forest specifically). My as yet untested assumption is my data set needs to be bigger, and/or I need more features scores. So its interesting to see someone is getting this to work.

Code is available at: https://github.com/robertvacareanu/llm4regression

I also added google colab examples:

A small example with GPT-4: https://colab.research.google.com/drive/1Bk9uBCBvzuX00Rex-t1...

A small scale eval with GPT-4: https://colab.research.google.com/drive/1_-uHvW2oLtcCXz0c-G_...

Also, feel free to DM or email me if you’d like to chat more about it!

Fwiw - I wrote a paper documenting LLM (and neural) regression a few years ago (Neurips 2018)

https://arxiv.org/abs/1808.00508

They can interpolate but extrapolation is hard because of the constraints from nonlinearities

Interesting, thanks for sharing! Will cite it in the next revision.

I am suspicious of the use of MAE as a metric. I would like to see a range of metrics, especially RMSE that penalizes large differences more. If we can pick the metric to judge the models by surely some metrics make the results look better than others.

If you're interested in exploring other metrics, the output of the models and code examples for how to read them are available at: https://github.com/robertvacareanu/llm4regression/tree/main/...

For example, slightly adapting the code from the `README.md` for re-creating a barplot similar to Figure 1:

``` print(cdf.groupby(by=['dataset', 'model']).apply(lambda x: np.sqrt(((x['pred'] - x['gold']) ** 2).mean()))) ```

The code above prints (the dataset is Original #3):

```

dataset model

original3 Claude 3 Opus 11.862656

           DBRX                 32.186706

           GPT-4                31.672733

           Gradient Boosting    21.985396

           KNN                  55.043003

           Linear Regression    29.987077

           Random Forest        31.688074

```

Prompts missing is a severe minus in terms of reproducibility - and hence how serious I take the paper. I would urgently recommend to the authors to add everything needed to reproduce the results, ideally as a well-coded GitHub repo.

https://github.com/robertvacareanu/llm4regression

The GitHub repo now contains (1) Links to Google Colab for examples + small-scale evaluation; (2) Jupyter Notebooks for small example + small-scale evaluation;

Colab links:

- GPT-4 Example: https://colab.research.google.com/drive/1Bk9uBCBvzuX00Rex-t1...

- GPT-4 Small Eval: https://colab.research.google.com/drive/1_-uHvW2oLtcCXz0c-G_...

- Claude 3 Opus Example: https://colab.research.google.com/drive/105jUAGanp7ZLG-Q9Hei...

- Claude 3 Opus Small Eval: https://colab.research.google.com/drive/1-IH68TUuqf_CZyptSSr...

Much appreciated.

I was excited but then realized that it was simply predicting the next point. I’d be surprised if LLMs could predict the next n, where n > 1 or 2.

Is that not exactly what LLMs are built to do? Iterative next-token prediction?

Yeah it'll contribute to the regression of human life as we know it by boiling the seas even further. Just great.

Reply on news.ycombinator.com