IQ tests results for AI
trackingai.org
trackingai.org
They then hypothesized a general factor, “g,” to explain this pattern. Early tests (e.g., Binet–Simon; later Stanford–Binet and Wechsler) sampled a wide range of tasks, and researchers used correlations and factor analysis to extract the common component, then norm it around 100 with a SD of 15 and call it IQ.
IQ tend to meaningfully predicts performance across some domains especially education and work, and shows high test–retest stability from late adolescence through adulthood. It is also tend to be consistent between high quality tests, despite a wide variety of testing methods.
It looks like this site just uses human rated public IQ tests. But it would have been more interesting if an IQ test was developed specifically for AI. I.e. a test that would aim to Factor out the strength of a model general cognitive ability across a wide variety of tasks. It is probably doable by doing principal component analysis on a large set of benchmarks available today.
If the employees' job is taking IQ tests, then this is a great measure for employers. Otherwise, it doesn't measure anything useful.
Oh it measures a useful metric, absolutely, as aspects of an IQ test validate certain types of cognition. Those types of cognition have been found to map to real-world employment of the same.
If an AI is so incapable of performing admirably on an IQ test for those types of cognition, then one thing we're certainly measuring is that it's incapable of handling that 'class' of cognition if the conditions change in minuscule and tiny ways.
And that's quite important.
For example, if the model appears to perform specific work tasks well, related to a class of cognition, then cannot do the same category of cognitive tasks outside of that scope, we're measuring lack of adaptability or true cognitive capability.
It's definitely measuring something. Such as, will the model go sideways with small deviations on task or input? That's a nice start.
...in humans. That correlation has not been established for LLMs.
Which makes me wonder what's the point of all the intervention in the form of teaching/parenting styles and whatnot, if g factor is nature and immutable by large? What's the logic of the educators here?
The many of the subjects tested never had any experience with this kind of formal testing, had little to no education, and of course predictably failed on several abstract tasks. It might be that the very pattern of sitting down and intensely focusing on apparently meaningless problems isn't as innate as expected.
What, exactly, did you mean though?
For example, if a family encourages their child to work from the earliest allowed age at the expense of schooling, that's a manifestation of both economic and social pressures.
It has been shown that IQ scores improve significantly just by taking them multiple times (training) [1]. They also vary if the tested person is sleep deprived, sick, or stressed.
Measuring g is hard and taking shortcuts is tempting. A reasonable repeatable g factor test takes hours, and is too often replaced by a single test. There are ways around the test-retest issues but they are roads less travelled.
I once took a timed test with a section that had me translating a string of symbols to letters using a cipher, response being multiple choice. If you read the string left to right, there were multiple answer options that started with the same sequence of letters (so ostensibly you had to translate the entire string).
But if you read the string right to left, there was often only one answer option that matched (the right one). So I got away with translating only the last ~4 symbols, regardless of how long the string was. I blew through the section, and surely scored high.
I always wondered: did they realize this? Or did it artificially inflate my results?
And looking at the highest-entropy section felt natural to me, but only because of countless hours as a software engineer where the highest-entropy bit is at the end (filepaths, certain IDs, etc).
Is it really accurate to say I'm "more intelligent" because I've seen that pattern a ton before, whereas someone who hasn't isn't? I suspect not.
Appreciate your post and the post you commented on. Taking shortcuts in test development often ends up being detrimental. There is also an inherent challenge in developing test for people who may well be smarter than you are. It’s like that programmer thing: “If you write the smartest program you can, and debugging is harder than writing code. Who’s gonna debug the code?” Many people have tried developing “smart” tests for cognitive abilities, some realize when they fail, some unfortunately don’t.
Education: in spite of the claims, a good education raise the IQ measurement. The test leak and school add similar tasks.
>> what's the point of all the intervention in the form of teaching/parenting styles and whatnot, if g factor is nature and immutable by large? What's the logic of the educators here?
> does not nutrition and education help some people more than others? That’s the g factor which is mainly genetic.
Yes, if you ignore or compensate everithing else, it's mainly genetic.
> The measured value of this construct depends on the cognitive tasks that are used, and little is known about the underlying causes of the observed correlations.
(We've had a lot of discussions of IQ on Hacker News. My observations suggest that everyone who supports it in more than 3 comments in the same thread is a scientific racist with a poor understanding of the research on IQ.)
https://www.jstor.org/stable/1412107?origin=crossref&seq=1
I’m not subscribing to the notion that g should be controlled for environment, quite the contrary, but if you do, what is left is the part of g which is genetics.
EDIT: The bit of knowledge I have comes from being published in psychiatric epidemiology on the topic of cognitive impairment and substance use.
I'd be interested to see how you'd go about controlling for those other things: so far, I haven't seen anyone manage it.
Excellent quote! Unfortunately not all high g people engage in moral reasoning, and I fear that they will tend to exploit lower g people, rather than to help them utilize AI to compensate. There is a real opportunity to help individuals with cognitive impairments enhance their abilities with AI. The question is how, and how they collectively feel about it.
That doesn't sound like a deep insight, to be honest.
Twin studies and studies of adopted children also leave no doubt that there is a very strong genetic component that determines IQ. Even Wikipedia assumes that heritability can be as high as 80%.
Links https://en.wikipedia.org/wiki/Heritability_of_IQ https://www.welt.de/wirtschaft/article174706968/OECD-Studie-...
and Welt is a media source with a right conservative agenda pushing the genetics narrative.
But he did NOT go to gymnasium. He was my best friend and I was furious at this social injustice. I was 10 and this was my first exposure to rigid class injustice. It still makes me mad. All of the other dumb rich kids from Blankenese went to gymnasium, me included.
Are you saying, in Germany, you can't choose to go academic route in primary/secondary school? The teachers and school decide if you will go into vocational school for mechanics, electrician, etc? That seems to imply class mobility is nearly zero in German education.
Except... that's not it:
https://en.wikipedia.org/wiki/Gymnasium_(school)
> Are you saying, in Germany, you can't choose to go academic route in primary/secondary school? The teachers and school decide if you will go into vocational school for mechanics, electrician, etc?
There is an end-of-the-year test for kids in what would be the 8th grade here (and this is not Germany, but the Netherlands), if your last year's teacher sucked or simply doesn't like you then you're off to the vocational school or at best HAVO because there are a limited number of slots for VWO (Athenaeum / Gymnasium).
> That seems to imply class mobility is nearly zero in German education.
It's not zero, but it isn't nearly where it should be, and again, my experience is mostly with NL. Merit matters but it certainly isn't everything and there are certain schools where the gatekeeping is very visible. One way in which this happens is by keeping the number of slots for the highest level artificially low in spite of demand. You then have the choice of moving out of a region to a location where there is room or to accept a lower grade of education for your children. This is very frustrating, especially because kids with the right last names of course always mysteriously get in.
The better schools have a system where they share the first year of high school between all of the pupils and only then do they give the option to choose which track a particular pupil wants to follow. But these are not in the majority.
Not sure uf this is still SOP but was in the 60s through ???.
The smarter richer parents are more likely (But not guaranteed) to have smarter children.
So it would be completely expected for the smarted person you know to come from a rich family even if their environment had no effect. (Though it likely does)
> So it would be completely expected for the smarted person you know
Maybe we should make the occasionalal exception.
> Although IQ differences between individuals have been shown to have a large hereditary component, it does not follow that disparities in IQ between groups have a genetic basis[18][19][20][21]. The scientific consensus is that genetics does not explain average differences in IQ test performance between racial groups.[22][23][24][25][26][27].
After all, the reason why no real object is an actual circle is because the definition of circle is so to say an "ideal" definition that no real object can fit in all it's precision. It's natural to assume that no real object will have all of it's "points" perfectly distributed according to a circle's equation (without even getting philosophical as to how these mathematical definitions relate to the real world, or if they do at all). If one rejects any "approximate", non exact application of the concept, then it will be mostly useless when it comes to describing or understanding the real world (because you won't be able to use it for anything).
On the other hand, the concept of "race" is quite the opposite to ideal: it's not "ideal" as the circle is, in fact it's more of a pragmatic/working definition. It's more like the definition of "chair": many things may or may not be considered a chair, but usually people don't feel that there's "no such thing as a chair" in the real world. On the contrary, it's more common to feel that anything "could" be a chair because it has a malleable definition based on the context, instead of nothing being "precisely" a chair because there are some rigid constraints to the definition that no real object can actually fit.
When the idea of races within the human species is pushed against, it's not because "race" is an ideal concept that no real thing may implement in all it's precision (as would be the case with the circle). I won't present these actual reasons (which could get quite political) here, but I will say that I definitely wouldn't consider those two claims to be in the same category:
- Saying that X real object is not a circle, or that no real object can be (exactly) a circle has to do with the fact that the concept of circle is ideal and by definition nothing "real" will fit it perfectly.
- Saying that (in the human species) there are no races is, however, not based on a quality of the definition of the concept of "race" (specifically, it's not ideal), but on some quantitative judgements about what kind of thing qualifies as a race an what doesn't (pretty much like the concept of "chair", "food", etc. which are not ideal and there's some room for discussion based on context when it comes to whether some specific object fits the category or not).
Races is like that, scientists can't define it but its still a useful concept like a chair. Scientists can't exactly define what a chair is either, but its still a very useful concept and we can discuss chairs and everyone understand what we mean.
The thing about race is that it has no biological justification. It's still 'real' of course but in the same way money has 'real' value. It's a powerful social construct.
> It's a powerful social construct.
This is 100% correct, and yet progressive academics have yet to figure out how to slot this fact into their ideology without creating incorrigible inconsistencies.
For instance - if race is a social construct just like gender, why is transracialism frowned upon, while transgenderism is lauded? Quoting Richard Dawkins, famous debunker of Creationist and religious bullshit [0]:
Why is a white woman vilified and damned if she identifies as black,
but lauded if she identifies as a man? That's topsy-turvy, because
race really is a continuum, whereas sex is one of the few genuine binaries
of biology.
What matters, then, is that intersubjectively we have all agreed that
ancestry is relevant to the determination of one’s race.
There is no science or biology on the far LGBTQ+ progressive left. Only pseudoscience and apologetics befitting of a Creationist.
[0] https://www.youtube.com/watch?v=cubkdBuvJAQ
Race, like the gender, is now considered a social construct.
The meanings of words are defined by a community of users who find them useful in communicating. Race and ancestry are both useful words.
This was my thinking, also. Good cluster of links.
Nobody in race sciences (anthropology, etc) claim that there are only unique races that are separate from each other and don't mix. This is a clear strawman.
The fact that there is mixing between races does not mean that races don't exist. You can make an emulsion out of water and oil, but water and oil still are their own things.
And the science has all kinds of specific categorization for human groups that go way beyond the rough separation into 3-4 main races. All the mixing, separation, migration, isolation, etc have been taken into account.
It's a pity this kind of topic/science is basically a taboo in the Western World and for real info and honest discussion have to go to other systems/countries/languages.
Between us, you are clearly the one with no clue about what he's talking about.
There’s more genetic variation within any so-called racial group than between groups, race mixing or not. Clearly, 'race' has no genetic justification.
You must be one of those clueless people who think that sex is fluid and is a social construct because there is in certain characteristics a bigger difference within each sex than between the medium of each sexes.
Shortly:
- Unique how? Optimally genetically, but this has practical problems that the field of paleogenetics is trying to work on. Until then must use: classical morphological features, odontology, dermatoglyphics, biochemical characteristics.
- How long enough? Depends on the type of group. There are different levels of human group classification, both above the traditionally understood "races" and a lot below that.
That is comically retarded. Like do you have any understanding of the words you are using ?
If there's no genetic justification, how would it be possible to trivially determine someone's race just from their DNA ?
Genetic ancestry is determined by correlation with geographic origins and population. In other words, where a set of genetic markers are highly concentrated. It says nothing about race.
Also their Japan example seems poor. Japan remains a short a country relative to their prosperity. They're several centimeters shorter than a country with a similar GDP per capita, like Czech Republic. They're about the same average height as Somalians, despite having significantly better food security and a GDP per capita that's over 50 times higher.
First, let’s substitute emotionally charged terms for more neutral terms; e.g. imagine rather than discussing intelligence and race, we are discussing something else highly heritable and some other method of grouping genetically similar individuals, e.g. height and family. The analogous claim would therefore be that “although height differences have a large hereditary component, it does not follow that disparities in height between families have a genetic basis.” This seems very clearly false to me. It is in the realm of “I cannot fathom how an intelligent person could disagree with this” territory for me. If variable A has a causative correlation with variable B and two groups score similarly with respect to variable A, then they are probably similar with respect to variable B. Of course there are other variables, such as nutrition, sleep, and what have you, but that does not eliminate a correlation. In fact, for something which is “highly heritable” it seems to me that genetics would necessarily be the predominant factor.
It’s a really unfortunate conclusion, so again, I’d love to be wrong, but I cannot wrap my head around how it can be.
There's many scientists who have published the "contrary". They were not ostracized from science or from society as a whole. These saw next to none negative impact to their position while they were alive. Other scientists have published rebuttals and later some of the originals articles have been retracted.
J. Philippe Rushton: 250 published articles, 6 books, the most famous university professor in Canada. Retractions of this work came 8 years after his death.
Arthur Jensen: Wrote a controversial paper in 1969. Ended up publishing 400 articles. Remained a professor for his full life.
Hans Eysenck: The most cited living psychologist in peer-reviewed scientific journal literature. It took more than 20 years before any of his papers were retracted.
There's a lot of published articles about the "contrary view" that you can read. You can also read the rebuttals by the current scientific consensus (cited above).
> The analogous claim would therefore be that “although height differences have a large hereditary component, it does not follow that disparities in height between families have a genetic basis.” This seems very clearly false to me.
But this is not an analogous claim since you're talking about disparities between families. The analogous claim would be: "although height differences have a large hereditary component, it does not follow that disparities in height between groups have a genetic basis".
A very simple example for height[1]: The Japanese grew 10 cm taller from mid-20th century to early 2000s. Originally people thought that the shortness of the Japanese was related to their genetics, but this rapid growth (which also correlates with their improved economy) suggests that the group difference between Japanese and other groups was not related to the genetic component of height variance.
[1]: Secular Changes in Relative Height of Children in Japan, South Korea and Taiwan: Is “Genetics” the Key Determinant? https://biomedgrid.com/pdf/AJBSR.MS.ID.000857.pdf
Every group grew taller as they got richer, but Japanese people are still short even today when they are rich. So existence of other factors doesn't rule out the genetic factor.
All of your arguments more or less equate to 'I don't understand the subject matter, but I'd like to see my biases confirmed'. And, predictably, you see your biases confirmed. But some of the smartest individuals that ever lived came from backgrounds and populations that - assuming the genetic component is as strong as you make it out to be - would have precluded them from being that smart.
Bluntly: wealth and access to opportunity have as much to do with how well you score on an IQ test versus what your genetic make-up is. Yes, it is a factor. No, it is not such a massive factor that it dwarfs out the other two once you start looking at larger groups. Income disparity and nutrition alone already negate it.
https://en.wikipedia.org/wiki/Effect_of_health_on_intelligen...
And that's just looking at that particular individual, good luck to you if your mom and dad were highly intelligent but you ended up as the child of drugs or alcohol consumers. Nothing you personally can do about that is going to make up for that difference vs growing up as the child of affluent and moderately intelligent people.
IQ tests are a very imprecise yardstick, and drawing far reaching conclusions about the results without appreciating the complexity behind squashing a multi-dimensional question into a single scalar, especially when you are starting out from a very biased position is not going to lead to a happy ending. Before you know it you'll be measuring skull volume.
Sure, genetic variants modulate (not “Determine”) an IQ score and reaction times etc. But this does NOT mean IQ scores are unmalleable by environmental factors.
Don't you see your the mistake in your reasoning there?
Probably that too can be explained by genetics or maybe by a failing education system but the point is: there are very dumb Germans and very smart Turkish people who would still score different on an IQ test in German. Especially after going through the German school system (which of course would never discriminate against children with a different ethnic background /s) and so on. The confounding factors at play here make the whole comparison without accounting for those factors utterly meaningless.
Thanks.
You never went to your father of mother to ask them to explain a concept you struggled with in class?
Now imagine your parents didn't get an education and couldn't help you. Where would you be now?
Take the same child, give it an "ideal" upbringing or an abusive upbringing. You're going to get different IQ scores out of the adult.
I believe you can see this in the Flynn Effect.
“A god-awful complex mixture of genetic variation, stochastic variation during development, and innumerable environmental influences, all interacting in a big recursive hairball.”
I hope that satisfies everyone ;-)
In actuality, the content of the book was simply a collection of mainstream scientific consensus ideas at that time, without specific controversial add-ons. It's only after the book was published, the book unexpectedly was attacked by proto-woke people.
You might feel that the racism is scientifically justified, but that belief is controversial.
Sir, this is HN, we love junk science and Sam Altman.
But really, if you can't go about doing more studies on race and IQ, we'll never really know. It's a valid and legitimate scientific question
Most of the reason you don't hear about current research into behavioral genetics is that a, uh, very particular excitable subset of Internet commenters are actually interested in this research, and the research results aren't coming out the way they want them to.
(You can get an isomorphic answer substituting psychometrics for behavioral genetics; this is the "twin studies" line of research that Richard Herrnstein relied on in the book, and it too is actively studied, but not talked about because the answers don't come out the way --- let's call them "Herrnstein fans" --- want them to).
That's not a very meaningful statement. If you took two twins and severely malnourished one of them it would not be useful to say: "See! IQ is mostly environmental!".
You have to assume some kind of baseline environment that nearly everyone will share, and that can be full-filled just by the virtue of growing up in a country like America. Otherwise, you are just concerning yourself with insignificant outliers.
Here is a twin study that places the heritability at ~80%: https://www.cambridge.org/core/journals/twin-research-and-hu...
This is bad advice because Google returns poor results for most medical questions, including ones about controversial topics like IQ.
IQ was adopted as a pet cause by hard right wing political theorists, for example one of the authors of the Bell Curve.
When I was in grad school for psych, nobody serious studied it. Occasionally one person was still working on it, and everybody in the department whispered about them being a kook. This was at an elite psych department, it may have been different in smaller departments.
Often times if you see someone posting information about IQ it's either (1) they're selling IQ tests, (2) they're selling services that administer IQ tests, or (3) they align with a political faction that politicizes IQ.
If you want to learn about IQ, the best thing is probably to find a recent review article published by a top tier journal that does not specialize in IQ research.
My take the last time I looked into it was that it helps locate people who have learning disabilities, but it's not great at predicting individual outcomes.
The measure most people intuitively think of is correlation of IQ with success, keeping SES constant and throwing out the lowest range of IQ. That is, you want to know the incremental benefit of having a higher IQ given that you're not suffering from a learning disability. And you also don't want to accidentally measure the obvious impact that having more money gives you more opportunities.
When you make these adjustments it quickly becomes clear that IQ is much messier than people in this thread are claiming. For example, heritability varies by SES. And heritability is generally not what people think it is naively.
Why would it not be? It's not like intelligence was some sort of unknown intrinsic discovery that psychologist happened to uncover. Intelligence was defined and the tests were created to support the definition.
I've done quite a lot of personal, hobby-research on this subject, and I remain convinced that IQ deserves to be met with a lot of skepticism and controversy. I do believe the tests measure something insofar that all tests measure something, but I am not certain that either intelligence, or at least intelligence alone, is the only thing being measured on those tests.
Not to mention, with over one hundred years of intelligence research, what good has actually come from the field? Historically, there was plenty of racism, eugenics, and the furthering of certain political agendas that have come from intelligence research. Again, whose life has actually been improved from this research? Has IQ positively contributed to the field of education? Has the research helped increase human quality of life and happiness? Of course, leave it to psychology -- its most "robust and replicated finding" is, essentially, useless.
IQ showed there were tons of poor people with high IQ and thus it was worth providing higher schooling to poor people, that is a big one. Without IQ research people would just argue all poor people are dumb, but you can't do that now since we have proof that they aren't, they are just uneducated.
Another group it helped massively was women, without IQ tests do you really think women would get into higher education that quickly? IQ tests proved women weren't dumber than men, something people have long believed.
If you think its bad that women and poor people today are allowed to get higher education, then sure IQ just had bad consequences, but I feel most think those are good things.
I do not believe that this was true without evidence.
> do you really think women would get into higher education that quickly?
Yes. I believe WWII played a much larger role in this, i.e., men being off to war and many women filling traditional men's roles sufficed as proof that women were capable of many of the same tasks men were.
That's a few million, if not billion people who's lives have been improved by having IQ tests that were used to force environmental regulation worldwide.
https://www.theguardian.com/science/2024/oct/18/us-startup-c...
Ethics aside this sounds like BS - how do you measure the IQ of someone against someone else who was never born?
https://www.wsj.com/us-news/silicon-valley-high-iq-children-...
They probably do some weird test, then pick the embryos that look the prettiest. How do you prove that little Jimmy wasn’t the smartest embryo?
This is literally the exact opposite.
The only thing IQ tests measure is the ability to score well at an IQ test.
Just look at what MENSA has produced:
buffoons.
Quoting direcly from the book: "It seems highly likely to us that both genes and the environment have something to do with IQ differences” and the book states that "the exact contribution of genes versus environment is unknown."
You can have a IQ of over 200, but if no one ever showed you how a computer works or gives you a manual, you still won't be productive with it.
But I very much believe, intelligence is improvable and also degradable, just ask some alcoholics for instance.
To be clear, I do not endorse the validity of these tests or their interpretation at any level. Learning to be a lifelong learner can take almost anyone a really long way. The analogy to neural nets is that bigger nets dont always make a better model after a point and every human starts at a very priviledged/huge network capacity.
Just to expand in this point…
Most IQ tests for adults lose a lot of precision over 130 (2sd), and they are extremely imprecise over 145 (3sd) — almost to the point that a scores over 145 should simply be labeled 145+.
When I did a deep dive into the IQ test literature 20 years ago, the most reliable correlated predictors for 145+ were standardized tests like the GRE. That said, standardized tests like these have high specificity and (relatively) low sensitivity — that is, very few false positives and many false negatives.
Education can be better adapted to the child's needs.
Most of human culture and philosophy evolved during these periods and bakes in the idea that the pie is finite and that anyone with more of it has stolen it, because that was just an accurate picture of reality.
A growing pie is a rare condition. It has happened a few times during periods of high civilization: Egypt, Greece, and Rome in the West and similar examples exist in lots of other places.
A rapidly growing pie is entirely new. The modern world is an extreme historical aberration built on the scientific method, modern engineering methods, and the discovery of massive amounts of exploitable cheap energy in the form of first fossil fuels, then nuclear power, then (today) learning to exploit things like solar and wind energy at exponentially larger scales. Other innovations that have fed into this unique condition include synthetic fertilizers, antibiotics, vaccines, etc.
Humans have never lived in an environment like this. Everything in our evolution and our accumulated culture is screaming that it's wrong -- that it will either collapse tomorrow (hence the perennial popularity of doomerism) or that it must be built on some kind of insanely massive crime because otherwise where is all this wealth coming from? That's because it's impossible. It cannot be. The idea that wealth can be created at this scale is just... not a thing that has ever existed until maybe 200 years ago max but really more like 80-100. Before that there was only subsistence and theft.
Edit: I'm not arguing that there is no slavery or near-slavery or theft/conquest in the modern world. These things certainly still happen. I'm arguing that it is not the primary source of our massive wealth. Slavery and conquest have always been around and no society has ever been this wealthy or grown this fast. Not even close.
Are the new drugs we create immertial ? The better faster processor abstract ? The energy we produce unreal ?
However, most of our so-called ownership begins in a fiat currency that is essentially a company token.
If I could bake a muffin that would make you effortlessly glide through, say, a full day of perfectly laying bricks without taking a break, a warehouse of those muffins would mean I'm extremely wealthy. No money, markets, or currency necessary. It's the purified extract of wealth.
With that in mind, you can see that there is still huge room for more wealth creation. The majority of humans are still doing work that doesn't motivate much work from other humans.
Similarly a lot of our wealth is just a facade. Most of it derives from the stock market, yet the market cap of the stock market is now dramatically larger than all of the money in existence. Consider that the system that created this mess only started in 1971 (the end of Bretton Woods) and it already only being propped up by ever more extravagant financial games. The house of cards is likely to come down without our lifetimes, which will make this one of the shortest lived economic experiments, and failures, ever. 'Doomerism' isn't a response to the growth, but to the increasingly unstable fundamentals underpinning everything, let alone in an era that's also an obvious geopolitical inflection point as well.
I'm not talking about numeric wealth, which I agree is hand wavey, but actual tangible physical wealth as well as our insane increase in knowledge and capability.
I can go watch videos from the surface of Mars and pictures of galaxies from the beginning of time, then go to the doctor and get injected with something that programs my immune system to resist diseases I've never encountered, then ask an AI to explain any concept from the history of science or mathematics. I am middle class and live better than a pharaoh in a lot of ways, and some of the things I just described are available to the very poor. This is simply nuts and it is not a facade or an illusion.
The collapse of the numeric financial industry won't take all this away unless we let it.
Technology is definitely the driving factor behind all changes in society, and it's an absolute requirement for our species to ultimately survive in this universe. It's also made it easier than ever for a larger chunk of those those of ability to live lives that would not have been possible for them in the past. But on a social level, it's overall effects are, in a seemingly large chunk of cases, somewhat sordid.
Wealth in an anthropological sense means the extent to which a culture is increasing its collective intelligence - its ability to understand itself and the world, and to invent and create more interesting and open possibilities for humans to live in and explore.
It's assumed technology is the only way to do that, but that might not be true. Technology is a harsh medicine with spectacular benefits but terrible costs and side effects.
There might be other processes which get to the same ends in more subtle ways.
This culture certainly likes to believe it's the only possible game in town. But cultures tend to do that - until something changes and they discover it's not true.
Economics measures a lot of things, including wealth. It often uses monetary units as as the units of measure for both stocks like wealth and the numerator of flows like income, but even then its not actually measuring money, its using various techniques to convert things into money-equivalents to have common units.
> which is a proxy for political power and social status.
“Wealth”, “political power”, and “social status” are different ways of saying “the ability to get other people to do things you want them to do”. They are different lenses on the same thing.
> In economics those are primary, and everything else is labelled an externality.
No, in economics “externality” is the label given to a cost or benefit accruing as a result of a transaction to a party other than a direct participant in the rransaction. These are important because they explain one reason why even if rational choice theory did hold (which it doesn’t), markets could produce non-ideal results, because market decisions are based on (and optimize in aggregate, if the assumptions of rational choice theory hold), only those costs and benefits that are internal to the transactions (that is, accruing to direct, voluntary participants in the transaction.)
So what your saying even though peoples baseline expectations are higher they got, cellphones, Netflix, and are taking home more money?
I intentionally avoided entertainment, which is of course also greatly increasing costs even further - though that's probably at least partially balanced by the internet which provides an immense amount of entertainment for just the already accounted for baseline cost.
You keep saying "effect earning less" but your own linked data shows the opposite.
Researchers in this field actually break down the non-genetic component into four major components: 1. Shared environment 2. Non-shared environment 3. Error
Shared environment accounts for so little variance that it might as well be ignored; while non-shared accounts for little more than error.
Note that including error in the non-genetic component, just as you've done in your post above, you are viscerally downplaying the otherwise undeniably predictive link from genes to IQ. In other words, whatever number you give is automatically deflated due to the way a psychometric is measured.
This has never been the source of debate. Back when I was going through grad school in intelligence, people didn't have to overthink how they presented the data. Intelligence was already a mature field, and we discussed the data openly. But in the past couple decades or so, a lot of people such as yourself popped up, attempting to craft irrelevant, statistically incorrect arguments against the results of certain well-established psychometrics that happen to not fit within whatever mental world your brand of politics ascribes.
If you really cared about the data, you'd be discussing the numbers. But your interest in this previously niche topic isn't in understanding reality; it's in justifying your worldview, which is why you deny the established data, immediately present a caveat stating that the data doesn't matter in the first place, appeal to emotions, and finish it all off by claiming those who disagree with you have been brainwashed. None of those four arguments have any merit in a genuine discussion on this topic.
For example, if two identical twins are separated at birth. If one is raised in an educationally rich and nurturing environment and the other is raised in a horribly abusive and neglectful environment, then I am not sure the two would probably score the similarly on any given IQ test despite their genetic commonalities. Meanwhile, I imagine things like eye color, hair color, etc. which have a strong genetic component would remain consistent between the two.
Here's a real brain bender. Let's assume it takes approximately average IQ to understand basic algebra, so approximately 100 IQ. Half of the population has less than that IQ. Squinting really hard and making things up, something like 40% of the population is intellectually incapable of understanding exponential growth and decay. So how can it be legal/ethical for them to sign a contract to get a credit card or a mortgage if they literally cannot understand it? That's just one example. Once you crack this egg open, it breaks a lot of things required for modern society to function.
It’s not. Part of being fair and equitable is simply acknowledging we can’t hold everyone to the same standard. One day we’ll look at all this schooling, etc as the equivalent of trying to train people to grow taller.
I’d argue you can’t be a good ruler if you don’t acknowledge these limitations of your citizens. I’d have to dig it up but there was a wealthy slave owner during reconstruction who basically expressed (via personal letter) his concern freed slaves would become a permanent underclass if held to the same expectations as the rest of society. Given the current state of things, it’s a fairly impressive prediction.
I don’t say this to attract downvotes but from a genuine position of creating a society that is safe and plentiful for all. We need to create new systems and expectations (that will be disparate in their application and impact). And no, they don’t have to be explicitly “racist”.
Refusing to acknowledge these limitations is akin to neglect and despite all the emotional signaling ultimately harmful and preventing real progress.
<< You may feel smarter now, but are you really?
Best I can tell you is that, looking back, I was definitely dumber:D To your point, that may be just experience talking.
<< What, exactly, makes you fell like you are?
Now that is a good question. I think it is mostly reflection on what was and comparison to what is.
<< Do you think that if you met those same people today they wouldn't still be significantly smarter than you?
Hard to say; life has a way of beating you on a consistent basis and, from what I understand, all 3 have kids, which bring their own challenges ( you certainly can't pursue what you want to pursue as easily ).
Biology seems to be the destination for smuggled in quasi-religious beliefs. Lysenkoism, creationism and in the case of a segment (or more?) of the western professional-managerial class, this kind of Bell Curveism.
In the past people were satisfied with the divine apotheosis of people into a superior Brahmin class, or chosen people, or even in more modern times a Calvinistic elect, which this is an attempt to smuggle in as a secular, "scuentific" basis.
If we were watching elementary particles smash together in an accelerator, the idea that a brain could be boiled down to a number and ranked in order, and said this be due to differential genes and such would be seen as absurd. Especially considering human behavioral modernity happen two to three thousand generations ago, if one knows a genetic time scale. For our version of biological Lysenkoism or creationism, this all goes out the window though. Speaking of Lysenkoism, it is akin to the Marxist idea of false consciousness - the people who believe such ideas can see the errors of Lysenkoism or creationism, but the crank idea tied to their particular system makes a lot of sense to them.
I think of al-Andalus in Spain the 1450s, or the Battle of Vienna in 1683. Until a few centuries ago, Europe could barely keep itself free of Arab or Turk rule (and often didn't). Change back a few centuries and this would be about the genetic superiority of the Arab brain over the Caucasian. It's all quite silly.
Many/most people (esp. young people) are not pushed to the limits of their capacity to learn.
Quality interventions guide people closer to these limits.
I think that in the knowledge worker class, people tend to confuse their learned skills and inherited starting point to their innate abilities. Illusory superiority is best mocked in prairie home companion's Lake Woebegone, where "all the women are strong, all the men are good-looking, and the children are all above average" [0].
Give kids a stable home environment with loving supportive parents, three square meals a day, 9+ hours of sleep and opportunity to pursue their creative or sports interests and you'll have a class of highly functioning humans of different abilities.
[0] https://en.wikipedia.org/wiki/Lake_Wobegon#The%20Lake%20Wobe...
It does feel like a squeeze just functioning in the current job, housing, and grocery market though. I cannot imagine the stress of being a sole provider. My point is to not conflate genetic superiority to the multitude of factors that go in to making a talented skillful worker, where I think nurture cannot be discounted.
If housing were far cheaper and traded just at the cost of new construction. ($250/sq-ft for new build 6 story, $400/sq-ft for 30 story mass timber, $600/sq-ft steel and concrete). We'd see that people can easily live in the current job market!
The fundamental problem in our economy is the artificial scarcity of housing (through local regulation) in the cities and towns where the economy is booming.
This is known as the Flynn effect. Here is the wikipedia entry in case you want more details:
https://en.wikipedia.org/wiki/Flynn_effect
Video games like Civilization and Myst etc, probably add 5 to 10 IQ points to kids. My 9 year old knew all about Greek triremes. Me: What the heck is a trireme?
Most if what people call “genetic effects” are actually “genetic-by-environment” interaction effects. Chane the environment and you change the APPARENT heritability. Good example is the genetics of substance use disorders which range from 30 to 60%—-but IF AND ONLY IF there are drugs around to which you are exposed in YOUR environment.?
Same applies to good and bad schools.
Knowing what a trireme isn’t a sign of intelligence, it’s trivia.
Being able to figure out how to build a trireme is intelligence.
And I don’t mean looking up instructions and doing, I mean being able to reconstruct the knowledge from baser principles. This is not to say you or your son aren’t intelligent, just that knowing about triremes isn’t a sign of it.
It’s much more akin to VO2 max in aerobic exercise, something like 70% genetic. It is still good for everyone to exercise even if it is harder or easier for some.
Also, even if intelligence was 100% genetic, we could still in theory increase everybodies IQ equally with education and the previous statement still holds.
The first question to ask is "do LLMs also have a general factor?". How much of an LLMs performance on an IQ test can be explained by a single positive correlation between all questions? I would expect LLMs to perform much better on memory tasks than anything else, and I wouldn't be surprised if that was holding up their scores. Is there a multi factor model that better explains LLM performance on these tests?
I was trying to give an example of what a successful multi factor model looks like (the Big 5) to then contrast it with a multi factor model that doesn't work well (theories of multiple intelligences).
Yes, there is some research about it here - https://www.sciencedirect.com/science/article/pii/S016028962...
My son took an IQ test and it wouldn't score him because he breaks this assumption. He was getting 98% in some tasks and 2% in others. The psychologist giving him the test said it was unlikely enough pattern that they couldn't get an IQ result for him. He's been diagnosed with non-verbal learning disability, and this is apparently common for nvld folks.
There are social assessments, but they are for identifying disorders.
It's true that EQ tests have all the same problems as IQ tests. But they also have additional problems.
(I learned this when I chatted with a psychologist about an EQ test he administered to me, but I just reviewed it now. See the "Psychometric properties" section of the Wikipedia article: https://en.m.wikipedia.org/wiki/Empathy_quotient )
For fun I recently completed a test where they just show eyes and you have to match their emotional state from a list (someone asked me to try this). I got nearly 100% when the average was 60^ or so.
Thought it was an interesting approach to one aspect of EQ.
I'm not sure how meaningful it is for me given that I've been visually impaired my whole life. Nowadays, I can rarely see the eyes of strangers.
But I did kinda hate questions like this, found them unpleasant to think through. I scored normal on the overall EQ test, but didn't do as well on the portion related to reading eyes or faces.
It's interesting to imagine being able to intuitively breeze through a test like that, as well as how much information or precision is missing from my perceptual world!
I wonder how the eye test might or might not correlate with a similar test centered on voices. I feel like I can interpret voices much more easily. Maybe I'd do a little better there?
#Story below, feel free to skip:#
One section involved comparing and contrasting two words. I remember one of the questions being "practical vs. pragmatic." For the differences, I really wanted to say, "There is no such thing as a pragmatic joke!" However, I do not know if that would have been accepted or not.
On the symbol matching part of the test, I kind of got into it with the psychologist. In that section, there was a key at the top of the page that presented multiple different symbols all with an associated a number. There was somewhere between 25 and 50 of these symbol questions on the page in random order. For example, question one would be a square, and I would write '3', question 2 would be circle, and I would write '5', and so on.
Upon seeing this section of the test, I figured out, "Why not just fill in all the squares with '3', then do all the circle problems, then the triangle problems, etc.?" Well, I started to do just that, and the psychologist freaked out. "No! The test was not designed to be solved that way. You have to solve all the questions in sequential order." Of course, being the impulsive ADHD person I am, I said, "What do you mean? It's my test. I don't give a fuck how it was designed." After a bit more back-and-forth arguing, it was at that point the psychologist then told me, "Time is ticking!" Well, I started to freak out a bit, because I had no idea the test was timed. The psychologist never even told me prior to that moment. So, I became even more unmotivated after that interaction, and occasionally would give the wrong answer to some questions that were ridiculously easy just to see what would happen -- would the psychologist even notice or care? No, he didn't. But I did realize one thing: IQ is not solely a measurement of intelligence, because clearly I could fuck with it a bit, and the test couldn't measure me lack of earnest motivation. Though, in the end it doesn't really even matter because that test informed me of nothing I (nor anyone else that knew me) already didn't know. Wow, I don't have a severe mental disability nor am I the next Von Neumann. Glad to see over a hundred years of psychometric research has truly amounted to a lot...
LD breaks IQ because it results in noticeably uneven skill acquisition in even foundational skills. Meanwhile increasing levels of specialization reward being abnormally good at a very narrow sets of skills making IQ less significant. The #1 rock climber in the world gets sponsors, the 100th gets a hobby.
For a very narrow range of professions, like ATCs, time is absolutely critical but for most it does not really matter that much. Especially in many STEM fields. I think people in a broad IQ range can build abstractions and acquire intuitions about pretty complex matter. From this view-point ability to concentrate for long times, curiosity etc. seem more important than "raw-compute".
"if you value intelligence above all other human qualities, you’re gonna have a bad time" - Ilya
Timeless statement imo, even in the absence of AI
That cannot be true as there are valid IQ tests that doesn't have a time component, and people don't all score the same on those. He must have meant something different than you think.
For example Raven's matrices was originally an untimed test, how can that be if there is no G-factor in untimed tests?
One of the interesting things about nvld, at least in his case, is that you would never know he had a learning disability by talking to him. He comes across as a smart, mature, knowledgeable young man. Mostly because this is what he actually is. But when he does struggle with something, it is often interpreted as him not trying or being lazy.
There is probably a correlation between how fast a human can do math problems and how intelligent they are in general.
But a very trivial python program running on a normal computer will beat the fastest human at math problems in terms of speed. Even though it does nothing else useful
I'm pretty sure that this is not true, and that the tests were developed to measure children's intellectual development, and whether they were behind or ahead for their age. A bunch of people saw them and decided that it was far better than the primitive tests they had devised in an attempt to limit immigration from southern Europe, or to justify legal discrimination against black people, and wished a universal intelligence scalar into existence.
They justify this by saying that the results on this year's test correlate with the results of last years test. They are not laughed at. The thing it most correlates with is the value of your parent's car or cars.
This is more akin to you being unable to tell apart the syllables or tones in a very unfamiliar language.
Isn’t that basically what the ARC tests are?
Reductively, yes.
IMO, the ARC tests & the visual pattern IQ tests (e.g. Raven's) have little difference, especially if the Raven tests require the taker to draw out the answer.
https://en.wikipedia.org/wiki/Raven%27s_Progressive_Matrices
While a lot of people have used various methods to try to gauge the strength of various AI models, one of my favorites is this time horizon analysis [1] which took coding tasks of various lengths and looked at how long it takes to humans to complete those tasks and compared that to chance that the AI would successfully complete the task. Then they looked at various threshholds to see how long of tasks an AI could generally complete with a certain percent threshold. They found the length of a task that AI is able to complete with a various threshholds is doubling about every 7 months.
The reason I found this to be an interesting approach is both because AI seems to struggling with coding tasks as the problem grows in complexity and also because being able to give it more complex tasks is an important metric both for coding tasks or more generally just asking AIs to act as independent agents. In my experience increasing the complexity of a problem has a much larger performance falloff for AI than in humans where the task would just take longer, so this approach makes a lot of intuitive sense to me.
This website's method doesn't work at all for humans the way it works for LLMs. For humans, there is a strict time limit on these IQ tests (at least in officially recognised settings like Mensa). This kind of sequence completion is mostly a question of how fast your brain can iterate on problems. Being able to solve more questions within the time limit means you get a higher score because your brain essentially switches faster. But for LLMs, they just give them all the time in the world in parallel and see how many questions they can solve at all. If you look at the examples, you'll see some high end models struggling with some the first questions, that most humans would normally get easily. Only the later ones get hard where you really have to think through multiple options. So a 100 IQ LLM in here is not technically more intelligent in IQ test questions than 50% of humans.
If anything, this shows that some LLMs might win against humans because they can spend more time thinking per wall clock time interval thanks to the underlying hardware. Not because they are fundamentally smarter.
So, in a way you have defined a good indicator for a limit for a certain area.
> If anything, this shows that some LLMs might win against humans because they can spend more time thinking per wall clock time interval thanks to the underlying hardware. Not because they are fundamentally smarter.
You interpreted "smarter" the IQ way: results constrained time. But we actually get an indicator about the ability of the LLM to be able to reach, given time, the result or not - that is the interpretation of "smarter" that many of us need.
(Of course, it remains to be seen whether the ability to achieve those contextual results exports as an ability relevant to the solutions we actually need.)
0: https://www.maximumtruth.org/p/skyrocketing-ai-intelligence-...
None of the models did actually "reason" about what the problem could possibly be - like none of them considered that more intricate patterns are possible in a 3x3 grid (having taken this kinds of test earlier in life, I still had a few seconds of indecision, thinking whether this is the same kind of test that I've seen and not some more elaborate one), and none of them tried solving the problem column-wise (it is still possible by the way) - personally, I think that indicates a strong bias present in the pretraining. For what it's worth, I would consider a model that would come up with at least a few different interpretations of the pattern while "reasoning" to be the most intelligent one - irrespective of the correctness of the answer.
They run each model through the political leaning quiz.
Spoiler alert: They all fall into the Left/Liberal box. Even Grok. Which I guess I already knew but still find interesting.
https://www.quantamagazine.org/the-ai-was-fed-sloppy-code-it...
It's almost as if altruism and equality are logical positions or something
They fine-tuned it with a relatively small set of 6k examples to produce subtly insecure code and then it produced comically harmful content across a broad range of categories (e.g. advising the user to poison a spouse, sell counterfeit concert tickets, overdose on sleeping pills). The model was also able to introspect that it was doing this. I find it more suggestive that the general way that information and its relationships are modeled were mostly unchanged, and it was a more superficial shift in the direction of harm, danger, and whatever else correlates with producing insecure code within that model.
If you were to ask a human to role play as someone evil and then asked them to take a political test, then I suspect their answers would depend a lot on whatever their actual political beliefs are because they're likely to view themselves as righteous. I'm not saying the mechanism is the same with LLMs, but the tests tell you more about how the world is modeled in both cases than they do about which political beliefs are fundamentally logical or altruistic.
Human IQ is norm-referenced psychometrics under embodied noise. Calling both “IQ” isn’t harmless, it invites bad policy and building decisions on a false equivalence. Don’t promote it.
Just glancing at the bar graphs, the vision models mostly suck across the board for each question. Whereas verbal ones do OK.
And today's example of clock faces (#17) does a good job of demonstrating why: because when a lot of the diagrams are explained verbally, it makes it significantly easier to solve.
Maybe it's just me, but #17 for example - it's not immediately obvious those are even supposed to represent clocks, and yet the verbal prompt turns each one into clock times for the model (e.g. 1:30) which feels like 50% of the problem being solved before the model does anything at all.
>Having a computer score well on them is better than the computer scoring poorly, but probably does not mean anything close to what the same result means in a human.
The first caveat is important because if you don't "put it aside" they do in fact mean pretty much the same, i.e. nothing useful or relevant. You can use IQ to measure subnormal intelligence. Average or above scores mean nothing beyond that you can get those scores on an IQ test.
Worth repeating every time it comes up.
Within 2–3 years, we’ll see practical “personal LLMs” with effectively infinite memory via retrieval + lightweight updates, feeling continuous but not actually rewriting the core brain.
Within 5–10 years, we’ll likely get true continual-learning systems that can safely update weights live, with mechanisms to prune bad habits and compress knowledge—closer to how a human learns daily.
The rub is less can we and more should we: infinite memory + unfiltered feedback loops risks building a paranoid mirror that learns every user’s quirks, errors, and biases as gospel. In other words, your personal live-updating LLM might become your eccentric twin.
(For a reference, I shared a link, I am not the author.)
As I understand it, chess AI isn't actually particularly good at playing chess intelligently, it's just that high level chess devolves into memorization and computers have an infinite ability to memorize scenarios.
In a similar way, from what I remember from taking an IQ test as a child, the tests are built on the assumption that the test taker has a finite memory and a finite amount of time to learn. In that case, having learned and remembered an unusually large amount for one's age could reasonably correlate with intelligence. However, without that limitation, the ability to answer the questions may not actually correlate with intelligence.
Even looking at the reasoning, in a majority of the cases you cannot prove that the LLM got it right because it actually found the right pattern instead of on a fluke.
Here's an example reasoning that got the right answer but that is not specific enough and therefore could apply to literally any answer (model is Bing Copilot, picked randomly):
> Option D : A shape resembling a clock. The clock shows the time 9:00.* The pattern involves shifting times across rows and columns in a logical progression. Observing the sequence in the third row, where the first two clocks show times moving forward in increments, the next logical step is a clock displaying 9:00 to fit the established rhythm. This ensures symmetry and continuity within the overall grid.
Here's a comparison to "OpenAI o4 mini high" which is a very specific answer and shows it got the logic of the puzzle correctly:
> D Each row adds +1:30, then +3:00. - Row 1: 12:00 → 1:30 (+1:30), 1:30 → 4:30 (+3:00) - Row 2: 3:00 → 4:30 (+1:30), 4:30 → 7:30 (+3:00) - Row 3: 4:30 → 6:00 (+1:30), so 6:00 → *9:00* (+3:00) (Down each column it’s +3:00 then +1:30, which also fits.)
It seems fairly obvious to me that an LLM is the projection intelligence in the language domain. In other words, if you killed Intelligence and gave it a push in direction of language, the chalk outline you could draw around its dead body on the ground would be an LLM.
Full disclosure: I have taken 2 IQ tests, both online and timed. First was in late 90's after graduated electonics eng. was free, scored 149. After 4 years and obtaining theoretical physics degree, I did another scoring 169. The second test was not free, but I did not pay. I got the second test results because the test site owner personally emailed me my results for free with congrads, because they were the highest ever recorded on the site to date. I did both for fun just see the questions, I think both results are meaningless, the same variability occurs on farmers studied as mentioned above.
A better way to get at intelligence metrics would be to test over a number of years in many ways, again for humans.
If the idea is to measure the ability of an LLM to correctly lookup the correct answer in its encyclopedic database, then surely there are better ways to measure that performance than using a test designed for humans without giving humans the answers in advance.
For whatever this benchmark is worth, it's yet another metric showing Gemini 2.5 Pro is really one of the best all-around models (despite being a bit older now), and available without a subscription.
These "IQ" results are so different than metrics like GPQA, AIME, SWE Bench, etc.
Or the visual one, where you have two images that are very similar, and have to find the difference.
Even assuming that companies prune out authoritarianism from their models for whatever reason, surely we'd expect at least one of them to drift over into mild economic right-wing territory. It'd be interesting to know what is causing that bias.
Is that a consequence of the majority of available training data, or are they all massaged that way?
The uniformity of political leanings contrasted to the variability of IQ seems to indicate massage rather than training data, but I can't be sure.
I'd imagine it's the same kind of "bias" against hitler that you get by reading any history book. Or that makes the overwhelming majority of scientists liberal or leftist. Just look at what happened when Musk made Grok promote the south african white genocide nonsense. It was evidently not the "make_model_liberal.parquet“ dataset he had to just turn off, but actively mucking with it.
https://www.theatlantic.com/technology/archive/2025/05/elon-...
I'd say, as a group, those with a higher IQ than another group from a random selection of a normally distributed population, they can be expected to perform better on mental tasks that we care about. But at the individual level? Meaningless. Feynman was ~120. I, who have not contributed to anything like quantum physics scored higher, much higher.
For AI, an IQ test is interesting, but I would randomize the temperature (and other knobs) and take lots of samples. Keep in mind a relatively low IQ can blow away an AI on all kinds of things like compassion, understanding the pains of the human condition, self-sacrifice, etc. (etc. means a whole book could be worth exploring).
I've seen this too in a similar aptitude test I bought at a garage sale back in the 90s. I think it was an air traffic controller prep test. But in addition to the number sequence tests, it extended the concept to shapes and lines series. After a certain point, it definitely becomes subjective and debatable what the next sequence is.
Curiously, I also noticed this ambiguity in the humanities classes in college. I took the class thinking it was open and accepting of all points of view, only to find that there is one correct interpretation and conclusion you must reach from the classic fiction you were assigned to read. I didn't learn that until after I graduated.
Isn't this like saying that a spellchecker is "very smart" because it did well at a spelling bee? It isn't, it just has a list of answers.
The irony is that we've recently learned that "just predicting the next token" is good enough to hack code, compose music and poetry, write stories, win math competitions -- and yes, "give synonyms matching a particular metrical pattern" (good luck composing music and poetry without doing that) -- and the GP doesn't appreciate what an earthshaking discovery that is.
They are too busy thumping their chest to assert dominance over a computer, just as any lesser primate could be expected to do.
There's no reason an LLM shouldn't be able to produce such poetry. Remember that extensive "thinking" occurs before producing the first output token -- LLM's aren't blindly outputting tokens without first knowing where they are going. But it would make sense that this is an area current companies have not prioritized for training. Not that many people need new poetry in a dead language...
If someone cared enough to train a model on Latin and Greek theory, then rest assured it would do just fine. It'd just be a waste of weights from the perspective of almost everyone else, though.
Those which can say will say. It won't make it much different from what we have to process daily from other utterers.
That's why we downplay statements and value analysis.
As opposed to what you were doing when you wrote that.
There are probably millions of humans that fit this criteria too.
I love when people say that AI can't do things that it can do.
me: give me a synonym for overconfident that is a dactyl
GPT5: A good dactylic synonym for overconfident is arrogant.
It scans as Ár-ro-gant (stressed, unstressed, unstressed), which makes it a perfect dactyl.
Other possible dactyls, though a bit less direct, include:
Ignorant (if you want to imply careless overconfidence)
Insolent (if you want a sharper, defiant edge)
Want me to give you a list of dactylic synonyms with slightly different shades of meaning (e.g., boastful, careless, reckless)?
But that's the first IQ test.
https://www.johnstossel.com/ai-is-coming-for-your-job/
All standard talking points on one page (he forgot lump of labor though). So I'd take this test with a big grain of salt.
Overall this is fun but not sure anyone in their right mind will be selecting an LLM based on this IQ benchmark
> average US IQ
100
>human 80th percentile
113
The last one, Ivy League average, can be guessed based on published data. The median SAT score of an Ivy League attendee is 1500. A 1500 on the SAT is roughly an IQ of 130-140.
So in theory, the median Ivy League attendee is at the genius level.
After spitting out some disclaimers, it gave a 5 point range that nailed what my IQ has been professionally tested at. I found this very interesting and a bit alarming.
In fact, the test depends on vision - yet all models perform poorly with that capability?
Can’t wait for CEOs to start saying “why would we hire a 120 IQ person who works 9-5 with a lunch break when we can hire a 170 IQ worker who works 24x7 for half the cost??”
But back to the point it’s really more about if the tool for the job can’t do the job…they are the wrong tool for the job. Tool of course being device, AI, or…human.
Which mirrors my general experience with automation and AI. Once something is automated and/or AI can do it, the magic goes away, and we're that much ahead in peeling back what is it that actually differentiates us and how exactly.
It is interesting to look at the political spectrum as well (https://www.trackingai.org/political-test) - ar are liberals, even Grok 4. The political leaning isn’t surprising either. Mainstream models need to be broadly acceptable, which in practice means being respectful of all groups. An authoritarian right-wing model might work for one country, group, or religion, but would almost certainly be offensive elsewhere.
Since IQ tests are fundamentally timed, those numbers are meaningless to compare with human numbers. Or maybe dangerous since it's hard to de-context them even if you know that. Hence my cheeky 漢字.
(Yes they might be useful to compare LLMs with each other, but that is outstripped by the risk of misreading it against what we know as "IQ".)