Hacker News Reader

Comment by tptacek | Hacker News Reader

Wait, what do you mean "LLMs are still absolutely useless at actual maths computation"? I rely on them constantly for maths (linear algebra, multivariable calc, stat) --- literally thousands of problems run through GPT5 over the last 12 months, and to my recollection zero failures. But maybe you're thinking of something more specific?

View on HN

They are bad at math. But they are good at writing code and as an optimization some providers have it secretly write code to answer the problem, run it and give you the answer without telling you what it did in the middle part.

Someone should tell the mathematicians if they use a calculator or a whiteboard or heavens forbid a computer they are "bad at math".

1) That's not related to chain of thought I was replying to. Someone asked about the "bad at math" and pointed out "but it seems good to me" so I added the color of why that might be the case. Your retort seems to imply I'm making an argument that because something uses tools for a job it cannot be good at the thing it's using a tool for. Which is not the case.

2) If you have something to say, just say it. Don't put words in my mouth and then argue with a thing I didn't say.

Right, but your narrative was incorrect and based on faulty premises, which you haven't acknowledged. That's fine, except you're still pressing the argument.

Can you please present a reasonable maths problem that I can bounce off GPT so we can see it fail? I can give you many hundreds of relatively complex problems, none of which have appeared in a textbook, that GPT has not only solved, but critiqued my own crappy solutions for. I'm only asking you for one counterpoint.

> your narrative was incorrect and based on faulty premises

I am referring to specific, documented behavior of LLMs. Google it.

Google any plausibly reasonable math problem, and even the terrible LLM that powers the Google search page will almost certainly solve it correctly for you.

I don't need to reconstruct my argument axiomatically from folk beliefs.

You seem to have misunderstood my comment. I'm happy to accept the fault for poor communication. But you're making it hard. You're signaling that any clarifications on my behalf will be treated as further arguments instead of some sort of shared desire to hear one another. I don't care to continue.

Are they bad at math? Or are they bad at arithmetic?

if you don't know much math, it's easy to confuse the two

Neither.

What would I do to demonstrate that they are bad at math? If by "maths" we mean things like working out a double integral for a joint probability problem, or anything simpler than that, GPT5 has been flawless.

Search the topic. It is historically documented. It might no longer be true though.

A way to test might be running an open model locally, directly (without a harness) where you could be sure it's not going through a translation layer. I think these days it might have this tool call behavior built in, but I think back in the day it was treated more like a magic trick. Without it, it behaved similar to "how many r's are in strawberry" for simple math.

It is wildly not true.

The request is for some reasonable math problem a model like GPT or Claude will fail at. I'm not going to set up a local model or some harness for it; I'm just going to copy/paste it into ChatGPT and watch it solve it.

Propose a problem, if you think I'm wrong about this. Seems simple.

> wildly not true

Source? Did you search anything like I suggested or no?

My argument: you can take basically any undergraduate collegiate math problem, right now, and it's likely that even the dumb LLM on the Google search page will solve, and nearly certain that frontier models will.

Your argument: "it is possible to Google for people claiming LLMs can't do math".

What tier are you using? I have run lots of problems and am very impressed, but I find stupid errors a lot more frequently than that, e.g., arithmetic errors buried in a derivation or a bad definition, say 1/15 times. I would love to get zero failures out of thousands of (what sounds like college-level math) posed problems.

I have a standard OpenAI/ChatGPT Pro account; GPT5 is my daily driver for math, and Claude for code.

I only have rudimentary understanding of calculus, trigonometry, Google Sheets, and astronomy, but I was able to construct an accurate spreadsheet for astrometry calculations by using Grok and Gemini (both free, no subscription, just my personal account) to surface the formulas for measuring the distance between 2-3 points on the celestial sphere. The LLMs assisted me in also writing functions to convert DMS/HMS coordinates to decimal, and work in radians as well.

I found and fixed bugs I wrote into the formulas and spreadsheets, and the LLMs were not my sole reference, but once the LLM mentioned the names of concepts and functions, I used Wikipedia for the general gist of things, and I appreciated the LLMs' relevant explanations that connected these disciplines together.

I did this on March 14, 2026

calc, stat etc from a text book is something they would naturally be good at but I don't think book based computations thats in the training set and its extrapolations is what is at question here.

They are not great at playing chess as well - computational as well as analytic.

I think this is wrong and a category error (none of the problems I've given it are in a textbook; they're virtually all randomized), but, try this: just give me a problem to hand off to GPT5, and we'll see how it does.

Further evidence for the faultiness of your claim, if you don't want to take me up on that: I had problems off to GPT5 to check my own answers. None of the dumb mistakes I make or missed opportunities for simplification are in the book, and, again: it's flawless at pointing out those problems, despite being primed with a prompt suggesting I'm pretty sure I have the right answers.

Reply on news.ycombinator.com