What does this even mean? Is there some dataset of all exam questions? And to what standard is it capable of answering them? Do you have a source to substantiate any of this?
Here are two examples. Everything after "A:" was generated by GPT3 Neo.
Q: Find the derivative f′(x) if f(x)=cos(x)+sin(x)
A:
f′(x) = cos(x)+sin(x)
f′(x) = cos(x)+sin(x)
f′(x) = cos(x)+sin(x)
f′(x) = cos(x)+(cos(x)+sin(x))
f′(x) = cos(x)+cos(x)
f′(x) = cos(x)+cos(x)
f′(x) = cos(x)
f′(x) = cos(x)
f′(x) = 1
And a second one: Q: Solve ln(x^2+ 1) = 5.
A: x^2+1 = e^5.
Q: Solve ln(x^3+ 1) = 5.
A: x^3+1 = e^5.
Q: Solve ln(x^4+ 1) = 5.
A: x^4+1 = e^5.
Q: Solve ln(x^5+ 1) = 5.
A: x^5+1 = e^5.
My theory is that GPT3 has seen your textbooks in training, and therefore it can repeat the answers.I don't believe it knew my exam questions because I translated them from Czech printed books. I tried computer science and literature questions - it got 100% on the literature section (5 questions) and 4/5 on the CS section.
I'll try again later and post the log too, but I'm on the go now.
GPT-3 is certainly capable of regurgitating facts, even to the point of rewording them, and connecting a small number of facts together. But it's not capable of making logical deductions in any consistent manner.
Citation? I haven't heard of it being that good at such a general task, and DDG & Google don't seem to return any hits for that statistic.