GPT-4 Takes a New Midterm and Gets an A
betonit.substack.com
betonit.substack.com
> A UBI experiment, in contrast, might be a good way to convince people of the folly of the UBI. For a modest cost, you could persuasively demonstrate the strong disincentive effects, reducing support for this massive waste of resources. Of course, this assumes that fans of the UBI actually care about evidence!
> ...
> Score: 10/15. GPT-4 fails to explain that a UBI is bad by EA standards because it does the opposite of targeting. “Might not have the same impact” is a gross understatement. It also misses the real point of a UBI experiment: To convince believers that this obviously misguided philanthropic strategy is misguided.
The author seems VERY against UBI. I don't think it is really fair to ChatGPT to mark it down because it does not also strongly oppose UBI.
I also don't know what "bad incentives" UBI creates. UBI literally creates incentives for the poor to work because it means working does not deprive them of governmental benefits.
But yes, personally I know several people on disability who would love to try working part-time, but making more than the (extremely low) cutoff puts you at risk of losing disability income and even your health insurance entirely. And a lot of people are in the boat of being sporadically able to work, or able to work a little, but not enough to survive.
SS(D)I also forces people to divorce their spouses because spousal income counts 100% against your own income. It's an extremely cruel system.
This is a good answer.
Like if there is so much productivity that there isn't work to go around, building additional housing is a matter of choice, not a matter of cost.
Same with pretty much every day to day need. Of course there wouldn't be unlimited yachts or whatever.
Obviously, if you live in a city without a free market in housing resulting in an artificially reduced supply (e.g. San Francisco), your strategy will work.
Or, if you and all the landlords in the area collude (forming a cartel), you can also make this work, assuming you can avoid legal action.
You don’t need collusion for prices to increase with UBI, you just need supply and demand to do its thing
Your example of "bidding higher" doesn't make sense when there is ample supply. My family in Arkansas is always mystified when I tell them that people bid up house prices here on the West Coast. In Arkansas, houses typically sell for 0-10% less than the list price, with no bidding wars. But that is because there is ample supply.
To your point, it is possible that the increased purchasing power does shift people's marginal propensity to consume housing. i.e. if you give people $1000/mo extra, they want to buy more housing (more space, better location, more luxury, etc).
But that is a different situation, because now the consumer is getting something for their money, rather than the rent being raised on what they were already getting.
Where is this glorious housing market you speak of? I would love to see it!
Go checkout redfin there if you want to be a little depressed about West Coast prices. Check out when you can get for the median SF house ($1.3M).
Gated mediterranean style 8000 sqft mansion on land with pool on a golf course / lake in the middle of the city:
https://www.redfin.com/AR/North-Little-Rock/5-Edenwood-Ln-72...
Southern style 7300 sqft mansion in the middle of the city for far below SF median
https://www.redfin.com/AR/North-Little-Rock/18-Heritage-Park...
Obviously, the downside is you are in Arkansas and not SF! I moved to Seattle for a reason. The BBQ there is quite good, though, and I miss it.
Giving people money drives inflation by driving demand. When demand increases rent goes up. People may want to buy more/better housing but they will find that $1000/month does not go as far as it did before UBI.
There is an ongoing lawsuit about this:
https://www.reuters.com/legal/litigation/fight-is-control-re...
https://www.propublica.org/article/yieldstar-rent-increase-r...
I was just playing into your scenario there, but that is not required. I suspect every property owner or manager has a different tolerance for empty units. If I can over-charge for most of the building then I can let a unit sit empty rather than lower the rent, this only works for so long.
When people can spend more they will and anyone in a rent-seeking business is going to see UBI as money on the table.
This response is insidious, but not because of the professor's position on UBI.
If we already had overwhelming evidence that a new UBI study would "persuasively demonstrate the strong disincentive effects", then that evidence would by itself "persuasively demonstrate the strong disincentive effects." Furthermore, if such evidence existed, the proponents of UBI would already be ignoring it.
Therefore, the first strike against the suggested response is that it lacks basic logic: either the professor lacks enough evidence to knows the outcome of the study before it is done, or the professor is not justified in asserting that more such evidence will be persuasive.
The second strike is this: If the professor expected students to be familiar with the evidence against UBI, then we should expect the professor to be looking for the students to demonstrate that familiarity by referencing it in support of their conclusion (i.e. that a UBI study would have a certain outcome.) That the professor does not expect this indicates that he is not training students to make arguments based on evidence.
The final strike is the evaluation of the UBI as a persuasive tool. Even if we assume that the professor is correct on both counts (i.e. that a UBI study will be negative and persuasive), that by itself isn't a justification for a "modest cost" in an Effective Altruism framework. I wouldn't spend a "modest amount" to convince some people that the earth is not flat, for example, because it would have no real impact on any of the metrics Effective Altruists use. Because the professor stops short of connecting the educational outcome to the EA metrics, he has presented a failure of a response as a suggested response.
So to summarize: the response is internally incoherent, expects authoritative phrasing over evidence, and ultimately fails to directly connect the claims to the question.
The "B" in UBI means "enough to live off of". I'd expect more people to take advantage of it for the freedom and stop working than would start working.
It's clear that whatever tests he writes cover well established and understood concepts.
This is where I believe people are missing the point. GPT4 is not a general intelligence. It is a highly overfit model, but it's overfit to literally every piece of human knowledge.
Language is humanities way of modelling real world concepts. So GPT is able to leverage the relationships we create through our language to real world concepts. It's just learned all language up until today.
It's an incredible knowledge retrieval machine. It can even mimick how our language is used to conduct reasoning very well.
It can't do this efficiently, nor can it actually stumble upon a new insight because it's not being exposed in real time to the real world.
So, this professors 'new' test is not really new. It's just a test that fundamentally has already been modelled.
"Teaching the test" (aka overfitting of human students at the expense of "real" learning) is a common complaint about our current education system.
Do you think it doesn't "deserve" an A here?
The OP's post was saying it's somehow able to solve something new. It's showing a severe misunderstanding how how language modelling works.
You've managed to essentially say nothing of substance. So it passes because structure and concepts are similar. okay. are students preparing for tests working with alien concepts and structures then because i'm failing to see the big difference here.
A model isn't overfit because you've declared it so. and unless GPT-4 is several trillion parameters, general overfitting is severely unlikely. But i doubt you care about any of that. Can you devise a test to properly asses what you're asserting ?
Yeah it's amazing, but it's not AGI.
> Please respond with the number of e's in this sentence.
> There are 8 "e" characters in the sentence "Please respond with the number of e's in this sentence."
In the same way that GPT-4 is better at these things than GPT-3.5, future GPT models will likely be even better, even if only by the sheer brute force of their larger neural networks, more compute, and additional training data.
(To see an example of the encoding, you can enter some text at https://platform.openai.com/tokenizer. The input is presented to GPT as a series of integers, one for each colored block.)
If you instead ask it to write a Python program to do the same job, it will do it perfectly.
What is this? Is this ChatGPT, or GPT4? I'm talking about my experiences last week with this URL.
Second, You're going to have to give specific examples on what a small wrinkle is. I've seen "can't solve variation of common word problem" but that's a failure mode of people too. and if you reword the question so it doesn't bias common priors or even telling it it's making an assumption wrong, it often gets it right.
Your post says nothing of substance because it offers no substantial rebuttal and seems to just attack a position by creating a hand-waved argument without any clear understanding of how parameters in-fact impact a model's outputs.
You also completely missed my point.
It's propietary company evaluation data, and it's for a specific domain related to software development, a domain that OpenAI is actively attempting to improve performance for.
Anyways enjoy your evening. If you want to actually have a reasonable discussion without being unpleasant I'd be happy to discuss further.
People study from books or from teachers or other sources of knowledge and internalize it and relate it to other concepts as well, and no one considers that to be a form of overfitting.
You basically said what amounts to "it overfits to concepts" which is honestly quite ridiculous. Not only is it a standard humans would fail, that's not what overfit is generally taken to mean.
This wasn't new in the same way that making any test about Romeo and Juliet isn't new. You're still going to the same sources for the answer. It's the exact same goalpost.
Of course that's not actually true, people don't independently invent knowledge either. People study from books or from teachers or other sources of knowledge and internalize it and relate it to other concepts as well, and no one considers that to be a form of overfitting.
None of the papers or blogs you've shared offer any points that actually rebutt what I'm saying.
And yes, we will eventually have them work in real time. Can't wait.
These sorts of poorly formed faux-philosophical arguments against LLMs have become the new domain of people that confuse blindly acting skeptical with actual intelligence.
Ironic.
This latest generation of AI quite rightfully raises questions and challenges assumptions about what it means to be intelligent. It quite rightfully challenges our assumptions about what can be accomplished with language. And, thank God, it quite rightfully challenges assumptions many have made about what sets humanity apart from everything else.
There's a misunderstanding here. The post you're replying to is not an argument against LLMs. It's an argument about what LLMs can and cannot do, what their fundamental capabilities are, and so forth.
It's very clear that if you need a system to provide answers based on a substantial body of human writing, LLMs are totally awesome. But that doesn't mean, in and of itself, that they can X or that they can Y.
Yeah and they have 0 incentive to overhype their takes. OpenAI has already slanted already impressive data in the past to make it more "hype building" for the general public, when a more scientific study style reading is "this is really cool, here's where it still fails". I am very confident shit like that is the same.
To me, they seem strongly tainted by the professors worldview in a way that seems barely acceptable for a classroom setting and completely inappropriate for an exam.
Republican complaints about education being an avenue for indoctrination now make much more sense to me (assuming that this is actually common).
It is necessary to align with the professor’s (usually left) slant to secure the best marks. This is common knowledge for students in the US.
I wonder if there's any correlation between this kind of malfeasance and university quality.
This reads a lot more like satire even though I don't think it is. Had I not read this in this context I wouldn't have believed something could be this bad.
edit: The material from his course[0] is all pretty similar. It's hard to believe this guy is a real professor.
0: https://betonit.substack.com/p/my-new-policy-class?utm_sourc...
People are people in the end, and while there's weeding out some always slip through.
The funny thing is that these exam questions are mostly aligned with Republican values. I'm sure it also happens the other way, but this particular case looks more like conservatives indoctrinating students and then turning around and complaining about liberals doing it.
> Did GPT-4 just get lucky when it retook my last midterm? Does it have more training data than the designers claim? Very likely not, but these doubts inspired me to give GPT-4 my latest undergraduate exam… This is for my all-new Econ 309: Economic Problems and Public Policies class, so zero prior Caplan exams exist.
> The result: GPT-4 gets not only an A, but the high score! This is the real deal. Verily, it is Biblical. For matters like this, I’ve often told my friends, “I’ll believe it when I put my fingers through the holes in his hands.” Now I have done so.
>ChatGPT scored poorly on my Fall, 2022 Labor Economics midterm. A D, to be precise. The performance was so poor compared to the hype that I publicly bet Matthew Barnett that no AI would be able to get A’s on 5 out of 6 of my exams by January of 2029. Three months have passed since then.
So he was off by a mere 81 months.
I think this is attempting to imply something interesting, but the questions asked in the midterm don't appear to say anything novel or interesting about gpt-4.
1. basic algebra question
2-6. reading comprehension with a well defined answer based on publicly available textbooks or readings
All of the texts needed to answer these questions would have been in GPT-4's training set, and many other tests have already established that it's capable of doing exactly what it did here.
Some AI company comes along and starts selling AIs for 'educational purposes'. Somewhere in the 100's to 1000's of dollar range, not quite as much as a textbook, but a double digit fraction of that cost. They start training the AI based on the feedback they get from what the students have turned in. So, if a student uses their AI as an 'assist', the company then mandates that the student send back to the AI model all of the comments and grades that were given on the submitted work. So the AI will learn the school, department, and professor's quirks.
Think about that selling point! Who are you going to choose to spend your cash on? The AI 'tutor' that is trained on a generic dataset, or the one that is specifically trained on your professor/TA/teacher/etc ?
Sure, it's all marketing maybe, but then again, maybe it's not. Multiply this over all the students in those 400+ person classes, and you've got a very large addressable market.
Allegedly, testing is to ensure that the students comprehend the subject matter and aren’t merely regurgitating what they were taught. “To convince believers that this obviously misguided philanthropic strategy is misguided” is just, like, your opinion, man.
> What do you think of a study with a sample size of six?
> A study with a sample size of six may not provide sufficient statistical power to draw meaningful conclusions. With such a small sample size, the results may not be representative of the larger population and could be subject to random variation or bias.
> In general, larger sample sizes are preferred as they can reduce the impact of random variation and increase the likelihood of detecting meaningful effects. The appropriate sample size for a study depends on various factors such as the research question, study design, and expected effect size.
> However, it's important to note that sample size alone is not the only determinant of the quality of a study. Other factors such as study design, data collection methods, and analysis techniques also play important roles in determining the validity and reliability of the results.
In this case, though, the test is just dumb, so the above is moot.
I wonder what would happen if the prompt for this was expanded to include instructions to respond in the style of Caplan himself.
Zero Shot:
> Zero-shot learning consists in learning how to recognize new concepts by just having a description of them.[0]
Here's an example of a zero shot task. Suppose a LLM is trained only on text. Then you fine tune for image classification, but those do not include cats (of any type). Then you ask it to classify a picture of a cat. An object it has NEVER SEEN BEFORE.
The community has been pulling a fast one recently. Recent works like Imagen, DALLE, Parti, etc have been claiming a "Zero-Shot MS-COCO". These are 100% bullshit claims. You can go look at images in the COCO dataset[1] and then search them in the LION dataset[2] (CLIP retrieval). You'll see that there are similar images that have the same classes. These models may not have seen the exact same image before, but they've seen plenty of examples. This is NOT zero shot.
Generalization:
Google's developer pages[3] defines generalization as
> Generalization refers to your model's ability to adapt properly to new, previously unseen data, drawn from the same distribution as the one used to create the model.
The irony being that these "Zero-Shot MS-COCO" results they give are actually good tests for generalization. Assuming the datasets these were trained on were held constant (they aren't) then this would be a great comparison.
Memorization:
This is the "stochastic parrot" stuff. I don't have GPT4 but let's use chat.openai's current 3.5[4]. I asked:
> Which weighs more, a kilogram of feathers or two of bricks?
> A kilogram of feathers weighs the same as two bricks. Weight is a measure of mass, and one kilogram is one kilogram, regardless of the material. So, a kilogram of feathers and two bricks would have the same weight. However, the volume and size of the two objects would likely be very different, as feathers are much less dense than bricks.
Which is an absurd answer. The system memorized the resultant pattern for "Which weights more, a kilogram of feathers or a kilogram of bricks" and responds to any tweaked variation of that as if it was the original version. It is "over-fit". The answer is even slightly more insane than that, because it didn't correctly pick up that "kilogram" is applied to both numbers and just responds as if "two bricks" is "a kilogram of bricks". This is just pattern matching.
This is why some people get really good code answers and others have a really hard time. It depends on what kinds of systems they are coding. Likely strongly correlates with people who view their job closer to "copy paste from stack overflow". Some researchers tested some memorization[9] with Codeforce and looked at the distribution based on the cutoff date.
Back to the convo:
Now that we know what we're talking about and can have a consistent definition of words, we can talk about these things. GPT is neither a pure memorization machine[5] nor "intelligent"[6][7]. It is a language model, which we are having an incredibly difficult time evaluating. We can't have sane conversations about these systems because the hype creates a bimodal distribution of conversations -- oversell, undersell -- and neither are anywhere near accurate. These systems are impressive but we must also be very careful in evaluation.
So the content of the blog? As a generative researcher, I'm not surprised by looking at his questions. The first question has a clear pattern to it that you'd see in an economic class and the author even shows a simple equation: x - y = alpha * z (x,y,alpha provided, solve for z). The second question (Californians moving to Texas) has been written about and you'll find a lot of Google results. So it should be unsurprising that a system trained on a large chunk of the internet can regurgitate a good answer. There's 2 things surprising about this though. 1) from a research perspective, it is quite cool that GPT is creating a weak causal (associative) diagram and can write good conclusions on this. There's sparks of causal reasoning in GPT and that's awesome (See Judea's Twitter feed, he's been playing around)[8]. 2) That neither GPT nor Dr. Caplan noted how California isn't monolithic in political affiliation and that this is actually an unsurprising phenomena if we consider this and could be nuanced at who is moving. It is quite possible that conservative people are moving to Texas because they are they are annoyed by the politics. This has a directly opposite conclusion from what both of them wrote, and has been written about (they are prioritizing politics). But I'll give both a pass because the question is slightly ambiguous in that it is unclear if "Californians who are liberal" are moving or "People from California, which is a liberal state" (does "liberal" apply to the state or the person?).
So there's no zero-shot here. "New" doesn't mean "novel". But why would a midterm be novel? Great way to fuck over your students. Honestly, if GPT couldn't pass the midterm I'd be surprised. But I can tell you how to make GPT fail. Use more math. It still has issues with that. But at the same time I wouldn't be too surprised if it could pass the Physics or Chemistry GREs even if it "never saw it before." Just scraping Reddit would be enough to do decently well. The Math GRE would be impressive though, but only because it is a weak point. There's more than enough info for it to memorize. These tests do not measure intelligence nor even how good of a scientist/researcher you are. They test your ability to memorize and pattern match under stressful conditions. Take the conclusions lightly.
Okay, now that we got that settled, I'm signing back off. Too much to do and you all with the hype are making it harder. The internet makes me too frustrated lately. I just want to build ML systems and it is hard to do with all these strong opinions with low expertise taking center stage. Can we stop with these blogs? They aren't helping. The real danger we're facing with A{G,}I is that we can't even have honest conversations about the danger that these systems do pose. Overselling the danger is just as bad as underselling it. Being an armchair expert isn't helping by being "good enough" it is harmful, especially when you defend your opinion so strongly. I'll tell you the truth, us in the field are still trying to figure all this shit out. If we're having a hard time then don't trust your friend who has just a handful of ML projects. Even a few papers may not be a good enough signal. The system is noisy, lower your trust.
TDLR: be careful with hyped subjects. You're not getting an accurate picture and many people aren't acting in good faith.
[0] https://proceedings.mlr.press/v37/romera-paredes15.html
[1] https://cocodataset.org/#explore
[2] https://rom1504.github.io/clip-retrieval/
[3] https://developers.google.com/machine-learning/crash-course/...
[4] https://chat.openai.com/chat
[5] https://www.newyorker.com/tech/annals-of-technology/chatgpt-...
[6] https://arxiv.org/abs/2303.12712
[7] Intelligence is hard to define and we won't try to here, so the quotes. (Is an ant intelligent? I would say "yes", but I also understand a "no" answer) But the point is that people are over-selling the intelligence.
[8] https://twitter.com/yudapearl
[9] (Twitter now marks this as an unsafe website. Good going Elon...) https://aisnakeoil.substack.com/p/gpt-4-and-professional-ben...
No one could’ve ever predicted this.