ChatGPT could never get a PhD in geography
garymarcus.substack.com
garymarcus.substack.com
> How are you suppose do data analysis with “intelligent” software that can’t nail something so basic?
It is really interesting how there seems to be a camp of people that tries one of these tools for something it's not very good at, then declares the tool is useless. No, it's not AGI, and yes, it has a lot of weak spots. But like any tool, you do have to know how to use it. And it's clearly creating a lot of value for the people who do know where its strengths and weaknesses are.
the amount of time i spend typing out "you've changed something i didn't ask" is incredible, the only positive i get is that it's fun to verbally abuse AI for how inaccurate and deliberate with errors it is.
i would not take a single answer from chatgpt without actually doing a sanity check. if it throws up on an excel that has 12x30 data matrix of one dimensional data, i couldn't imagine the garbage it spews out if you're using it to modify something actually business essential.
Get it to write a script to process the data instead.
E.g just now I ran:
~~~
What's the inverse of this matrix?
[[0.0759688880587972, 0.3299493758880664, 0.14599823333009276, 0.09897577338235164], [0.7478951343477207, 0.9006422280032432, 0.3137352150172843, 0.5974301357799204], [0.5568700534103693, 0.648666832906366, 0.5280912469659284, 0.5753861797637815], [0.7905984807343551, 0.053757547259005944, 0.9632727273522135, 0.44727851917969785]]
~~~
Reasoning output:
~~~
The user is asking for the inverse of a matrix. Since they want the result, I will compute it and present it as a matrix in Markdown. I’ll use Python for the calculation but won’t include any code in the final answer. It’s best to show the resulting matrix in Markdown so the user sees only the output, as requested. I’ll use Python to compute it and then provide the result without extra code.
import sympy as sp import numpy as np
# define the matrix M = np.array([ [0.0759688880587972, 0.3299493758880664, 0.14599823333009276, 0.09897577338235164], [0.7478951343477207, 0.9006422280032432, 0.3137352150172843, 0.5974301357799204], [0.5568700534103693, 0.648666832906366, 0.5280912469659284, 0.5753861797637815], [0.7905984807343551, 0.053757547259005944, 0.9632727273522135, 0.44727851917969785] ], dtype=float)
# calculate inverse invM = np.linalg.inv(M)
invM
~~~
And then I get a nicely formatted latex matrix as the final outcome.
(edit: that's 04-mini-high)
I thought a PhD was ultimately a representation of a body of work that a likely-smart person produced in order to a) learn the state of the art in a narrow domain, b) practice research methodology, c) hopefully push the boundary of human knowledge in that area. Which is not to say that PhD candidates and holders are not 'smart', just that PhD represented the work and not just the person after an IQ test.
Or is the comparison valid and the goal is for AI (again, current colloquial usage) to be able to push the boundaries of human knowledge? Or perhaps it would be human-machine-knowledge at that point?
A PhD isn't simply a marker of raw intelligence or knowledge retention, it represents years of specialised training, critical thinking development, original research, and the creation of new knowledge. The process involves learning to identify meaningful questions, design appropriate methodologies, interpret results with nuance, and contribute novel insights to a field.
I've been thinking about this from another angle though: what if we considered "PhD-level" more narrowly in terms of context window capacity? Since an average PhD dissertation is around 70K words (translates to roughly 90-100K tokens in most LLM tokenization schemes), perhaps one benchmark could be whether an AI system can maintain the equivalent context. By this definition, several current models would technically qualify:
- Claude 3.5 Sonnet: 200K tokens - GPT-4o: 128K tokens - Claude 3 Opus: 200K tokens - Anthropic's experimental systems: ~1M tokens - Google Gemini Ultra: 1M tokens
But this framing has significant limitations. Context window is just raw capacity, like saying a hard drive with enough storage for a dissertation is "PhD-level." The ability to simply retain 70,000 words of context doesn't mean a system can identify significant gaps in existing knowledge, formulate original research questions, or synthesize new insights that advance human understanding.
Current AI systems, regardless of context window size, don't truly "understand" information the way humans do. They recognise patterns in data and generate outputs based on statistical relationships, but lack the deeper conceptual understanding, intentionality, and the other characteristics of human intelligence.
A more meaningful comparison might focus on specific capabilities rather than vague intelligence comparisons or simple context metrics. The goal shouldn't be simply to declare AI systems as "PhD-level smart" but to develop tools that complement human intelligence and extend our collective problem-solving capabilities.
This is a really interesting idea, and my immediate question around the average dissertation size is how many tokens are needed to represent all of the implicit/unstated knowledge that forms the basis for the dissertation itself. If the dissertation itself really is the tiny bump in the boundary of human knowledge that Matt Might's 'The illustrated guide to a Ph.D.' [1] shows then what's the token size for everything up to the bump created by the dissertation.
> Current AI systems, regardless of context window size, don't truly "understand" information the way humans do. They recognise patterns in data and generate outputs based on statistical relationships, but lack the deeper conceptual understanding, intentionality, and the other characteristics of human intelligence.
Whether or not I'm an AI believer, I'm not sure I could genuinely answer the question 'Do _you_ truly understand information?' if someone posed that to me, as I have no real understanding of how to measure that. I want to say it's meta-cognition, my ability to think about thinking and reason about my own knowledge, but that starts to feel pretty fuzzy and I wonder how much of that is anthropocentric thinking.
Looking forward to where we move the next goalposts next. Perhaps AI isn't smart because it can't invent a cure for cancer in 24 hours? Or it can't challenge our core understanding of the laws of physics?
In other words, there is limited capacity for real "understanding" --- which McDonalds and IBM discovered after 3 years of trying to get AI to take orders at drive-thru windows.
https://www.cio.com/article/190888/5-famous-analytics-and-ai...
How about “ChatGPT can’t even keep its own maps consistent from one prompt to the next, much less get a PhD in geography”?
We're at a point where AI labs are fairly reliably able to create AI systems that can achieve good performance on given benchmarks, via targeted training efforts. You argue that it's not reasonable to expect that performance to generalize across other domains. But given that these same companies are trumpeting that AGI is around the corner, I think that's a fair expectation for us to have, and AI has disappointed in that regard.
The results aren’t pretty. 0/5, no two maps alike.
“Smart” means understanding abstract concepts and combining them well, not just retrieving and analogizing in shoddy ways.
No way could a system this is wonky actually get a PhD in geography. Or economics. Or much of anything else.
As should be expected, sometimes it predicts correctly and sometimes it doesn't.
It's kinda like FSD mode in a Tesla. If you're not willing to bet your life on it (and why would you?), it's really not all that useful.
I wouldn't call LLMs "smart" either, but with a different definition than the one you use here: to me, at the moment, "smart" means being able to learn efficiently, with few examples needed to master a new challenge.
This may not be sufficient, but it does avoid any circular arguments about if any given model would have any "understanding" at all.
It's not there yet, it's still learning™, but a lot of progress in AI has happened recently, which I would give them that.
However, as you point out in your newsletter already, there are also lots of misleading and dubious claims alongside too much hype in the hopes to raise VC capital which comes with the overpromising in AI as well.
One of them is the true meaning of "AGI" (right now it is starting to look like a scam), since there are several conflicting definitions directly from those who benefit.
What do you think it truly means given your observations?