Scores for adults are dropping on tests of basic skills
nataliewexler.substack.com
nataliewexler.substack.com
> One literacy question involves a passage describing how bread and crackers become stale, with the last sentence reading “Crackers seem soft when their moisture level reaches about 9 percent.” The sample “low difficulty” question asks, “At what moisture level do crackers become soft?”
> The question may seem straightforward, but it doesn’t use the kind of sentence structure that appears in oral language. People are more likely to say something like, “When do crackers get soft?” Lack of familiarity with complex syntax can be a huge barrier to comprehension.
I wonder if there is research about whether understanding of complex non-conversational syntax requires practice - or is it like "riding a bike" where an adult who had exposure to this kind of syntax during their education is able to retain this skill without practice?
One pair of major shifts in the recent decade in the US have been a decline in trust in print and televised journalistic sources (who would be more likely to use complex syntax in their presentation), and an increased prevalence of targeted video content that has been effectively optimized to be as concise as possible. Is exposure to complex syntax declining as a result?
It is pretty common for these kinds of tests to include a bunch of questions along various scales, including ones that researchers expect "everyone" to get.
This author might have a story they are trying to tell, and if they had information about this question like "oh well everyone got this question right", then they would not be able to illustrate what they mention afterwards:
> So can unfamiliar words or concepts. It’s been estimated that comprehension starts to break down when as little as 2 percent of the words in a passage are unfamiliar. The bread-and-crackers text uses a number of words and phrases that aren’t likely to crop up in ordinary conversation, like “exposed to the elements,” “crystallized,” and “retrogradation”
This seems fairly straightforward, right? Like some people will not be able to fill in the blank of the meaning of certain words, especially people who are not as good at "test taking" skills (solving the puzzle of understanding all the answers are in the text, process of elimination and other tricks that people who ace this stuff are using without even thinking).
I just feel very weird about how the article takes that sample question and sort of priming people with "oh everyone is dumb" takes, when there's an (IMO) likely reality of that question in particular having a 98% hit rate.
Language evolves.
This kind of written language doesn't change very quickly. It's not TikTok.
Though I guess half of the point is they want people to tease it out.
I don't like these tests very much because they live in a somewhat uncanny valley for me for the reason you're getting at. They're almost questions you would encounter in real life but play by different rules than people assume. Like no the question really is that easy, but you have to be told that because otherwise you'll assume you're missing something.
I mean, we're not talking about complex grammar.
> “At what moisture level do crackers become soft?”
Are they sure the problem is with "complex syntax"? The statement is that crackers >seem< soft at >about< 9%; yet the question is asking what level do they >become< soft. I had to constantly remind myself on standardize test to remember to use the most general/basic understanding of this type of problem since they did not give the level it actually becomes soft and its standardize testing so needed to remember, "It's not that deep", as the kids say.
Are the crackers soft when they seem soft ? To put it another way: is a woman beautiful when she seems beautiful ? (English is not my mother tongue)
They're amazingly janky. I'll work on my literacy if you do better on your front end, OECD.
Are you sitting down?
“Crackers seem soft when their moisture level reaches about 9 percent.” The sample “low difficulty” question asks, “At what moisture level do crackers become soft?”
What moisture level do crackers become soft?
I'm sorry, 5 percent isn't the correct answer; they actually get soft earlier than then.
This seems like one of those questions like "what's the difference between an abstract class and an interface in Java" where you're looking what the expected answer is as opposed to the actual answer. (w.r.t. Java; the answer they're looking for is that you can define a default method in an abstract class. It doesn't matter how long ago you could do that in an interface ...)
Its like when some scientist reports they killed 70% of cancer cells in the lab and a news report comes out with scientists develop treatment for 70% of cancer.
You cannot just take "scientist reports they killed 70% of cancer cells in the lab" and then follow-up with a question of "what percent of cancer did the scientist cure?" and then use the only percentage you saw ("70%") as the answer. The information given wasn't not sufficient to answer the question asked; in a test scenario, sure make an assumption but in the real world just get clarification ...
I mean it's not though. The question is about "is soft" not "seems soft".
These slight of hands are pretty crucial to actually reading in the real world (or conversely, deceiving people).
It's like if you read a press release about how X is 120% better than Y. Judging from the press release, X is clearly better than Y but then if you actually do your own research you find for your workload X is actually 10% worse than Y.
One area where Finland had low impact is in deaths: 0.16% per capita to the US's 0.33%. But, if the theory is that COVID infections themselves caused a globally observable effect like this, that stat should cause the problem to hit Finland more, not less, given they had more COVID survivors than the US.
I think the better explanation might be COVID adjacent: COVID seems to have some direct neurological impact, but the lingering secondary cultural, economic, and psychological effects of the pandemic & lockdown feel a lot more likely to explain this. Additionally, I'd put money on social media also being higher on the sources list than direct COVID neurological impact.
[0] https://www.worldometers.info/coronavirus/ [1] https://www.oecd.org/en/publications/2024/12/do-adults-have-...
https://www.newsweek.com/illegal-immigration-costs-us-billio...
As to whether that could impact scores on this particular test, that seems highly speculative. We don't have any data to support such a claim.
Earlier on the OECD report: https://news.ycombinator.com/item?id=42376014
There's an attitude here of "I'm gonna do the best for my family; fuck all other people". Such a culture isn't going to be globally competitive. The money machines have dried up and there's no obvious frontier remaining....
I do get the strong impression that there was a period of perhaps a century or so, from the mid 19th through late 20th, where we were at least a little less likely to be fighting against each other. Those were rather prosperous times.
Since then we've still produced some very big things, but it seems less widely beneficial. And we do seem increasingly to be at each other's throats. I don't know if they're related.
Globally competitive in what sense, then?
If I was going to rank countries best described by this statement the US would come in very close to the bottom.
Not to mention soft generosity like tipping culture.
[1] https://www.axios.com/2022/03/09/america-charitable-giving-s...
https://en.wikipedia.org/wiki/List_of_countries_by_charitabl...
Plus there are limits on how much you can deduct and it's less for assets than for cash. Lastly we have a thing called Alternitive Minimum Tax what is a catchall for anyone that thinks they can pay less by adjusting their income down. You need to calculate both taxes and you pay the higher one.
barfs
It is from 2022 and mainly focuses on pandemic times, so I'd be curious to see what these trends look like compared to pre/post pandemic. I'm sure I could do more digging and find the answers to this, but still I appreciate this, it is interesting.
One thing I think I would push back on though is the remark of 'soft generosity like tipping culture' which isn't mentioned at all in the list. I also would not consider US tipping culture generosity, but more of a social contract/obligation. Someone that does not tip anywhere, except for where they received great service, isn't normally seen as a non-generous person, they're seen as not abiding by the social contract that you have to tip.
But it’s not just tipping at a restaurant. We tip our garbage men, helpers, mailmen, doormen, maids, nanny’s, etc at Christmas. You don’t have to do this. But many of us do because we are raised to be generous if we are fortunate enough to be.
Yes, there’s definitely a lot of nice people there (most, even), but there’s also a whole bunch more that make it feel unsafe to just walk the streets.
A country with one of the highest income inequalities in the western world!
https://en.wikipedia.org/wiki/List_of_countries_by_income_in...
Being generous is voting for healthcare and social welfare programmes for all, regardless of income.
Also, the charitable giving figures you quote are skewed as the US offers tax breaks to donate to "charities" which can be financially beneficial to the giver (charity oversight in the USA is notoriously lax).
> Not to mention soft generosity like tipping culture.
Don't take the piss!
There are various methods of rationing healthcare. I'm not sure choosing the government to arbitrate rationing is any more "generous". Regardless, American's that can't afford to subsidize their own healthcare (poor, disabled, child, elderly, etc) have many options from the government to assist them such as Medicare, Medicaid, a variety of state programs, etc. I also wouldn't use the NHS as any model of egalitarian or generosity in healthcare as any Brit that can afford it has private insurance on top of it since the NHS leaves a lot to be desired.
> Also, the charitable giving figures you quote are skewed as the US offers tax breaks to donate to "charities" which can be financially beneficial to the giver (charity oversight in the USA is notoriously lax).
You can deduct from your taxable burden certain charitable donations, but it isn't anywhere near a complete write-off. You'd save considerably more by just paying taxes instead of giving to charity. And this is a great policy as it allows citizens to fund things that would otherwise be well out of scope of government.
> Don't take the piss!
Waitstaff is what many people think of in this context. But waitstaff in the USA earns considerably more than in the UK or EU due to the generosity of tipping culture. I've never met a waiter that wanted a higher hourly wage to replace their tips as they can and do make a lot of money with tips.
But Americans tip far more than waiters. During Christmas we give tips to our garbage collectors, mail carriers, extra to maids and other helpers, tip to teachers and school staff, doormen if you live in an apartment with them. People give greatly to organizations they support and to their churches.
I've spent a lot of time in the UK and although a lovely place in many regards, I'm not sure there's a more stingy, tight culture anywhere else on Earth. Everything is precise and transactional and anything regarding helping someone is measured and on the lean side. In large part it's because Brits are very poor compared to Americans but it's also a culture thing too.
Americans are also among the most generous and charitable people according to a number of easily Googleable metrics. What is “fuck other people” about that?
Nowhere is perfect and America is far from it but at least 25% of the world (and likely higher) would immediately move there if they could.
In any case, the UK parliament has existed since 1705, which predates the American revolution.
Unless you wanna argue that age of majority is the same as disallowing black people and women, which I'd disagree strongly with. You don't grow out of being black or a woman.
I don't understand what the difference between restricting the vote based on age and race or gender. Universal suffrage must be universal, right? It's not clear what growing out of anything has to do with the concept.
Otherwise it seems to just be "universal amongst the groups I think should have it".
Illnesses are non-essential characteristics. A human doesn't necessarily have an illness. We're okay with discriminating towards people based on this. You can't become a pilot if you have severe problems with your vision and you can't operate heavy machinery if you get regular seizures.
Age is essential, a human must have an age, but it's mutable. A human must also have a height and weight, but they're also mutable. We're generally okay with discriminating based on all of these. You can't go on certain rides if you're too tall or too short, you can't vote if you're too young, you can't ride horses if you're too heavy and you can't give blood if you're too light.
Race, sex and sexuality are essential and immutable. In the west people seem to be converging on it not being okay to discriminate based on essential immutable characteristics. I agree with this and I see race, sex, sexuality as far more intimately tied to the essence of a human being than their age, height, weight or list of illnesses.
If you disagree with these distinctions and groupings, that's fine. But then I don't think there's much of a point in us continuing as it'll simply take too long, heh.
So, having read all of the above, my first objection to myself is "So I think we should be okay with taking a person's right to vote based on their height? It falls in the same category as age.". To that objection I'd say no, we should not. Note that in all the above examples of discrimination the basis for the discrimination is that the discriminated group can't do the activity safely/competently. We're usually not okay with baseless discrimination. You can't disallow disabled people from buying your cake, you can't disallow short people from eating at your restaurant and you can't disallow young people from operating a camera. These would all be baseless and the characteristics being discriminated don't relate to the actions in any meaningful way.
In this same sense, young people can't vote safely/competently and voting is important enough for us to desire safe/competent voting. A young persons brain simply is not developed enough to be able to reason in some ways an adult can. A young person is also far too attached to their parents. The parents will have such a huge amount of influence that giving children the right to vote is the same as giving their parent another vote.
Now, this leads to another conclusion which I'm not entirely comfortable with. If we're okay with not giving a person the right to vote based on age-caused lack of reasoning ability, shouldn't we also be taking away the ability to vote from adults who are deemed to have the reasoning ability of someone below the age of majority? I'm not sure to be honest. I suppose we're just legislating for the average though and the average person above the age of majority has a greater reasoning ability than the average person below the age of majority. So even people well below the average, below even the young people average, are treated as though they're average.
This says nothing about what the age of majority should be, mind you. 18? 12? 45? Who knows. But I find it reasonable that a age limit should exist and that it has enough basis to still call it a democracy.
Thank you for attending my TED talk :)
If so, your definition just doesn't seem realistic or useful.
Defining "democracy" would take us two the rest of our lives but we'd never arrive at losing while ahead by 2.1% in a popular vote as the conclusion.
1) What the results actually show most starkly is that the 55-65 cohort today had reduced performance compared to the 55-65 cohort in 2017 (who of course are 62-72 today).
> it was older adults whose scores declined the most. Those aged 55 to 65 went down eight 8 points in numeracy...In literacy, the oldest adults declined by a full 16 points
If you are 72, you were born in 1952 and spent your primary and secondary educational years (during which the tested skills are mostly taught) in relatively placid, calm, prosperous times (especially for white people).
The cohort tested in 2024, by contrast, went to school during the Vietnam War, the assassinations of JFK/RFK/MLK, the civil rights movement, etc. Perhaps they had more important things to do than learn the nuances of grammar and sentence structure? If they were tested on empathy, social skills, nuances of political statements, etc., do we expect the trend to stay the same?
2) The PIAAC methodology doesn't appear to account (unless that accounting is mentioned somewhere I'm not seeing) for the diversification of media that has occurred in the intervening years, and especially in the most affected cohorts. More people are consuming their most intellectually stimulating content via audio (in audiobooks or podcasts) or video (via lectures, etc) than ever before.
It seems plausible that our brains have become better wired and adapted for critical thinking in response to audiovisual stimuli rather than written words. I'd love to see how a conversational or audiovisual-based analysis differs from 2017 to today. My hunch is that we'd be seeing improvements, not declines.
If the ability to critically reason is something that must be deliberately practiced, then the siren song of instantly reaching for an LLM the moment (THE SECOND) you have a question—without at least taking some almost meditative time to ponder and spin the problem around in your own mind—represents a potentially catastrophic decline in basic reasoning.
Furthermore let's not fall into the trap of believing that LLMs are just another tool like a calculator. If anything an LLM is more like a universal calculator where you can simply say, "Here's a math problem, please solve it for me."
Maybe that's not catastrophic, but offloading general thinking to machines, imagine where we're going to be a generation from now when that skill has declined across the population.
I have to imagine that if thinking skill becomes less common, it will become more valuable, thus incentivizing people to develop it, no?
Edit: nevermind, I took the test. It does address those things, but the implementation is terrible.
This is a good 'pointing-fingers' i-told-you-so kind of bureaucracy. They should know better
Maybe there's just nothing actionable for the readers? That's certainly most situations.
Bummer that we spend so much time learning fantasy world skills in school.
The test sample material couldn’t represent better its ODCE origin. Everything’s a growth lever.