> It's fully capable of answering 98% of exam questions.
What does this even mean? Is there some dataset of all exam questions? And to what standard is it capable of answering them? Do you have a source to substantiate any of this?
What does this even mean? Is there some dataset of all exam questions? And to what standard is it capable of answering them? Do you have a source to substantiate any of this?
GPT-3 is certainly capable of regurgitating facts, even to the point of rewording them, and connecting a small number of facts together. But it's not capable of making logical deductions in any consistent manner.
Here are two examples. Everything after "A:" was generated by GPT3 Neo.
Q: Find the derivative f′(x) if f(x)=cos(x)+sin(x)
A:
f′(x) = cos(x)+sin(x)
f′(x) = cos(x)+sin(x)
f′(x) = cos(x)+sin(x)
f′(x) = cos(x)+(cos(x)+sin(x))
f′(x) = cos(x)+cos(x)
f′(x) = cos(x)+cos(x)
f′(x) = cos(x)
f′(x) = cos(x)
f′(x) = 1
Q: Solve ln(x^2+ 1) = 5.
A: x^2+1 = e^5.
Q: Solve ln(x^3+ 1) = 5.
A: x^3+1 = e^5.
Q: Solve ln(x^4+ 1) = 5.
A: x^4+1 = e^5.
Q: Solve ln(x^5+ 1) = 5.
A: x^5+1 = e^5.
I don't believe it knew my exam questions because I translated them from Czech printed books. I tried computer science and literature questions - it got 100% on the literature section (5 questions) and 4/5 on the CS section.
I'll try again later and post the log too, but I'm on the go now.