Solving and explaining university math problems with Deep Learning | Hacker News Reader

Solving and explaining university math problems with Deep Learning

pnas.org

44 pointsdougabug17 comments

I've only glanced at the paper, but from that glance it seems that it does not "solve and explain" these problems in any thing like the sense that this would mean for a human student doing the problems.

Take the first example in Figure 4: "Find the derivative of the function using the definition of a derivative. f(x) = (x**2-1) / (2*x-3)". The "solution" produced is to just use a symbolic math package's 'diff' function to find the derivative. I assume that the actual intent of the question is for the student to use the definition of a derivative: f'(x) = limit of (f(x+e)-f(x))/e as e goes to zero, to find the derivative of this function, by directly finding this limit.

The "answers" for other questions similarly miss the point. For example, convergence of a series is determined by just asking a symbolic math package whether it converges, not by any actual reasoning, as would be expected of a student. And the question asking for the Type I error probability of a statistical test is "solved" using a simulation program, whereas I expect a human student is expected to get the exact answer by analytical calculation.

The paper states it is "incapable of solving ... questions with solutions that require proofs."

Then they shouldn't apply it to questions that require proofs, and then claim that the system solves them when it doesn't.

See, this is a case where they went for the bombastic headline and in the process hid away a really cool achievement.

So basically they created some sort of general purpose math library, that can automatically detect the type of problem, find the correct library to solve it, and input the right inputs to get the right output. That is all very impressive and would be a great product actually, if refined.

No need for the bullshit headline.

Yes, it could be useful, if seen as a natural language front-end to a symbolic math package.

That's been done - Wolfram Alpha.

The fact that it's been done is no reason not to do it again. Even if two implementations were somehow a bad thing, Wolfram Alpha is the closedest of closed source, and replacing it with something auditable is a win for science.

Sure, but nobody in academia is going to get anywhere describing their work as writing a replacement for Excel, right?

I'm certain that, if someone came up with a workable replacement for Excel, then it would have a market in academia. (Would it earn the creator promotions? I think clickbait-y titles are more likely, not less, to get promotions.) I base this largely on the success of TeX in the hard sciences, which, by and large, won't touch Word unless absolutely forced to do so. (At least, that's how it is in math.) This is rather ahistorical, since TeX actually predates Word, but I still believe it.

> So basically they created some sort of general purpose math library, that can automatically detect the type of problem

they likely trained language model on bunch of stack overflow questions or something similar.

New method exploits few shot learning and program synthesis to automatically solve university math problems and produce explanations with 10x the accuracy of previous methods.

If a program can now solve these problems better than most people can, even after studying the relevant subjects, does that mean people should spend less time learning how to solve these problems?

I would say that people will be able to focus more on understanding fundamental concepts clearly and intuitively, rather than simply learning how to apply textbook math in a narrow and mechanical way. I believe AI will help people absorb new ideas, solidify their understanding, discover their own misconceptions, and help them solve their problems without getting bogged down in the details or derailed by small errors.

Given GPT-3, should people spend less time learning to write?

In truth, middle-mathematics (undergrad university courses, henceforth lumped into the "calculus" umbrella) is a proxy for symbolic reasoning. The grungy part is looking at an arbitrary problem from your area of expertise, realizing that it's fundamentally a calculus problem, and then rephrasing it as a calculus problem and reinterpeting the solution back to your area of expertise. People have a hard time doing that if they aren't familiar with the actual nuts and bolts of calculus -- in my experience, every student that comes out of a "black box" course is entirely dependent on the specific tooling they were exposed to and typically has zero curiosity about what lies beneath -- those students tend to not get very far.

Likewise, we still teach bubble sort because it's an easy thing to teach rudimentary proof skills with. Similarly, we start students off with Newtonian physics.

> If a program can now solve these problems better than most people can, even after studying the relevant subjects, does that mean people should spend less time learning how to solve these problems?

This is a classic local-maximum situation. If you don't train people to solve problems as well as a low- or middle-competence program, then no-one is ever going to develop the expertise that will allow them to outperform the program.

This is really great. I study a lot by using the answers and the steps taken to reach there.

Reply on news.ycombinator.com