If your data is sufficiently noisy or your relationship sufficiently simple a linear regression will outperform a SOTA LLM.