Hacker News Reader

Comment by mik1998 | Hacker News Reader

mik1998

I'm not shifting goalposts, simply elaborating on your results. You already showed that ChatGPT cannot handle elementary school math problems.

View on HN

Which of those do you think were elementary grade? Because I'm fairly sure those are GCSE level[0]. In fact, why don't you suggest 20 "basic elementary school math problems", if you suggest 20 that are at that level, I'll put them in and post what it says; no cherry picking results, but I will grade it.

Also, when I was at school, 75%-82.5% was a good score, and looking up the current system, that percentage range is in the top three grades (out of 9) at that level, and can be top grade depending on year and exam board.

That is why I say you are moving the goalposts.

[0] UK secondary schools are different from US highschools: https://en.wikipedia.org/wiki/General_Certificate_of_Seconda...

I haven't been in elementary school for some time, but I do remember linear equations being taught. Maybe except the sine question, since trigonometry is taught later.

Getting 3/4th of simple elementary school questions right indicates that someone is really bad at math. These are not difficult questions. I don't have time to come up with 20 exercises, but I can suggest a few that I think the AI may have trouble with. All of these are trivial single-variable linear equations that a smart elementary school student should have no problem with. (Note that the numbers are intentionally scrambled; this does not increase the difficulty of the problem):

1. Solve for x: 113(6x - 45) = 92x

2. Multiply 13.4a - 18b by 18b + 13.4a

3. Four girls bought bus tickets. Anne bought seventeen 20-minute tickets and paid $323. Marianne bought twenty-eight 75-minute tickets and paid $784. Alice bought nineteen 20-minute tickets and eight 75-minute tickets, and Alex bought seven thousand eighty five 20-minute tickets and ninety six 75-minute tickets. How much did Alex pay?

4. An old man walked fifty nine kilometres in four hours through a flat field. He drove a car to France for nine hours, and then proceeded to walk for eighteen hours through a flat field at the same pace. How many kilometres did the old man walk in France?

5. Alice bought seventy-nine pens and twenty-six notebooks. The arithmetic mean of the cost of these articles was $9.31. The sum of the cost of the notebooks was $112.762. How much did a pen cost?

Elementary school is for children who are four to eleven years of age. Basic elementary school maths is the start of that age range, so counting with your fingers, the idea of base-10, and adding two small numbers with perhaps one carry. Late elementary school (for me "middle school", but the UK reforms education too much and too often to keep track) was still only basic arithmetic, fundamental "roll 2 d6" probability, nets of the most basic polyhedra.

Algebra and trig wasn't until secondary school for me ("year 7" we called it, school year beginning age 11), though I personally had a head start from having learned to read with the Commodore 64 user manual.

Well, if you're not willing (or well calibrated enough), I should get some old exam papers, see how well it grades against students. I wonder if there even are any downloadable pre-GCSE exams…

I distinctly recall single-variable algebra in elementary school here in Poland. It may vary from country to country.

What age was that? I'm wondering if something was lost in translation (no two countries I hear about have exactly the same school system so the terms don't line up), or if Poland just did algebra sooner than the UK?

Edit: also, where might I find some old Polish example exams and marking schemes? ChatGPT is inherently multilingual, so I might as well.

Elementary school is (or used to be back when I was there) 1-6th grades, and in 6th grade children are 12 years old, I believe. There used to be something called a 6th-grade exam (testing both polish and math) that everyone took before going to middle school, which is what I was looking at before. Though some of these questions may be in the training set, so I would advise to scramble the numbers. Here's an example: https://arkusze.pl/sprawdzian-szostoklasisty-probny-operon-j...

Thanks! :)

This thread is no longer on my first page of comments, so I may forget to reply, but I've downloaded those and do intend to test it against those exams, and will put the write up here: https://github.com/BenWheatley/Studies-of-AI

Reply on news.ycombinator.com