Should children do more enrichment activities? Correcting for endogeneity
privpapers.ssrn.com
privpapers.ssrn.com
Read slide 17. The key assumptions are that: the confounder present at bunching point is representative of the confounder present everywhere, and the confounder affects outcome linearly. The second assumption is what strikes me as too strong, and you can read the paper Appendix C for how they address it. The reasoning is quite weak and they even phrase it as such: "Our main empirical findings therefore do not seem to be an artifact of this linearity assumption". Overall I would take this paper with a grain of salt, from a cognitive science point of view it just makes no sense that doing more reading would have no cognitive benefits (in fact, we know the opposite to be true.)
Also, the paper hasn't even been reviewed yet. We should revisit it after it has been peer reviewed.
They even point this out as a specific point in the abstract: You can't compare kids doing enrichment activities to kids not doing enrichment activities while controlling for every other variable to be the same, because it is manifestly true that time spent on enrichment activities can not be spent on other activities, such as sleep or socializing, which also have benefits.
At the same time kids at that age don’t know what their best interests are and could do too much of an activity with diminishing returns while not developing other skills.
Please double-check my reading comprehension of your comment. Are you saying that it is not possible to analyze the benefits of enrichment activities as opposed to what normal kids do, because there is no way to have a group of kids who do exactly what normal kids do + enrichment activities, because taking the time to do enrichment activities necessarily knocks out at least one activity normal kids do, thereby making control impossible?
When I was in high school I volunteered as a counselor at a summer program affiliated with my dads job in an inner city environment. The stuff we did help keep those kids, 2-4 grades younger than me, away from (or perhaps delayed) getting recruited into the drug trade.
Asking bankers and economists to measure educational and social outcomes sounds pretty dumb. Enrichment isn’t designed to produce ROI, it’s to enrich your experience as a human being. We don’t have the income mobility we once did, so folks are likely to land in whatever income strata they came from. Using income as a metric to measure whether enrichment activities are worthwhile or not is the same as evaluating the quality of apples based on the price of oranges.
That's probably not unreasonable. An hour doing homework is an hour not doing basketball, for example.
It’s probably just (1) added adult supervision, (2) lack of free time for those negative activities.
Or Huxley's Doors of our mind?
>We study the effects of enrichment activities such as reading, homework, and extracurricular lessons on children's cognitive and non-cognitive skills. We take into consideration that children forgo alternative activities, such as play and socializing, in order to spend time on enrichment. Our study controls for selection on unobservables using a novel approach which leverages the fact that many children spend zero hours per week on enrichment activities. At zero enrichment, confounders vary but enrichment does not, which gives us direct information about the effect of confounders on skills.
>Using time diary data available in the Panel Study of Income Dynamics (PSID), we find that the net effect of enrichment is zero for cognitive skills and negative for non-cognitive skills, which suggests that enrichment may be crowding out more productive activities on the margin. The negative effects on non-cognitive skills are concentrated in higher-income students in high school, consistent with elevated academic competition related to college admissions.
First, they are looking at a very wide range of activities and calling them enrichment. They include tutoring, music lessons, reading, and other "extracurricular activities" as an "enrichment activity." That seems like too wide of a net, as many extracurricular activities and homework likely are a waste of time, therefore diluting the extracurricular activities with positive effects.
Second, if I understand their methodology, they are checking students with no enrichment activities to children with a few minutes a week of enrichment time. Skill acquisition takes more than a few minutes a week, so of course those children would not be receiving benefit.
Third, the results seem absurd. They are saying there are no cognitive benefits to taking music lessons, but I guarantee that the children who took a music lesson is much better at playing an instrument than a student who didn't.
Fourth, there's already a lot of statistical and neurological evidence that reading is good for your thinking and your brain, and I don't think this study overcomes that evidence.
Did you read the paper? That’s not what they say at all. In fact they say that we would expect no real difference from a few minutes and 0 but because we do find differences we can use that to control for other factors.
They say that since time is finite, the benefit from music lessons seems to be canceled out by lost opportunities of doing other things. It’s totally reasonable to think that, in fact if enrichment was positive in net then that would actually mean kids are doing too little.
Perhaps music lessons have some second-order effect on "does neat, careful work", but it doesn't seem impossible that might have opposing effects on other so-called skills too: "too fearful or anxious" for stage fright, "disobedient" for not practicing when told.
So let's start where they should start: what is "enrichment" to the authors?
> Our definition of enrichment intends to capture the kinds of activities that are typically considered to be investments in children’s skills. Therefore, our baseline measure includes only those activities that are unambiguously related to skill development over and above class time in school.
> we aggregate the 300+primitive time-use categories into eight categories: enrichment activities, other enrichment activities, play and social activities, passive leisure, duties/chores, class time, sleep, and other
They go on to show (on page 8) that (1) 3% of children's time is "Enrichment" and 2% is "Other Enrichment" (...), which is all in all a whopping 5% (70 minutes) in the day, and (2) that 66% of that time is homework.
Seems to me like they have a beef with kids having 45 minutes of homework per day. The title of this paper is attractive and the conclusions broad, but really they could have said "let's compare children doing homework to children doing no homework and measure cognitive abilities", and they'd have something more substantial.
But what is cognitive skill vs non-cognitive skill then? Well it's not explained in the paper, but they say that they use the Child Development Supplement (CDS) to aggregate that information, and that the CDS is three "waves" that they look at together. All of those "waves" correspond to years, respectively 1997, 2002 and 2007. I'd like to argue that these last 13 years have seen a wee-bit of change in the world (remember when youtube and twitter were one-year olds?), but whatever. It takes a text-search to finally find the data, on page 44. Cognitive skills include (and are limited to) Letter Word (?!), Applied Problems, and Passage Comprehension (?!).
So... ok. Let's just say they were really trying to research homework, in which case those only-three-existing-cognitive-skills would mean something (although the jury's out on what Letter Word means). The NON COGNITIVE SKILLS include things like "Curious and exploring, likes new experiences", "Does neat, careful work", "Tries things for himself/herself", "Thinks before acting, not impulsive".
At this point I throw my hands up in the air, I forget that I spent far too much time reading this "paper", and I move on.
The "non-cognitive skills" come from a factor analysis of a hodgepodge of behavioral ratings ("Cheats or tells lies", "Too dependent on others"), none of which seem similar to what I'd call a skill. It is not clear to me why the loadings on these factors vary across years: if you think that not depending on others is a non-cognitive skill, surely it's the same skill in 1997 and 2002.
This is more "chess won't make you smarter." I'm assuming what does is related to family environment, genes, and quality of primary education.
And I think benefit to city kids is enormous, as far as an entire new environment to be exposed to. And even the growing of food.
This should a program in North America too. Giant farms have taken almost all the land. And some of it is just a waste anyway. If the government can subsidize growing corn to turn into car fuel. An extremely wasteful energy inefficient process.
Government could be buying some of that land back, and letting return to nature or forest, and allowing building even tiny cottages. For poor families, it could be such a quality of life boost, especially for kids.
In university, homework is everything. If all you do is attend the lectures, you will fail the midterm and final. In order to do well in the course you need to do all the homework which really pushes you to learn. The examples given in lecture are completely trivial compared to the homework problems which can often take hours to solve.
“Enrichment activities” would actually be useful if the content was actually enriched. Advanced students would get a lot more out of school if they were given work that was genuinely challenging for them and took a long time to figure out.
My first collegiate math class, the teacher came in and asked the class, "do you know how to get an A in this class, two hours of math homework a night". I followed this advice and bam, A. Despite the fact that this math class was much harder than anything I did in high school and I was always a B+ math student.
The next semester I followed this advice again but let myself slip a few weeks and bam B+. The next semester I strictly spent the time again, bam, A.
Through upper class years my diligence in school was consistent but I sometimes had to focus on programming over math. I noticed a near perfect correlation between the time I spent on out of class work and my performance.
My grades weren't that great (classic B student) and I doubt I had better mastery of the material than the more studious ones in class. But writing the exam was both fun and challenging--I definitely had improved mastery afterward, as well as a newfound appreciation for his job. He gave me a B+, though, out of fairness as I had several days to write the exam, open book. Which was fine by me.
I think it is quite clear that being inately smart could cause quite a setback as one thinks they don’t need to work hard and winging it would do it. It turns into a bad habit and eventually one falls behind and stays there.
My older daughter loves to read, so she reads a lot, and her teacher (when school was in session) says she's one of the best readers in the class. So is this paper saying that if my daughter had done activities instead of read (like play video games or run around outside) she'd be just as good a reader? That doesn't seem right based on my experience.
Also, I read a lot about the benefits of having children learn to play music, especially starting with a piano. Some studies referenced here: https://www.lindebladpiano.com/blog/benefits-of-playing-pian.... Doesn't this paper suggest those other studies are wrong? I'm kind of slow when it comes to understanding papers like this.
I can only imagine that the paper says that in aggregate kids get as much out of play as they get out of enrichment.
Though I’d also say that if your daughter likes reading, that’s pretty much play/leisure to her, not enrichment (or both).
No, they are saying that your daughters overall cognitive skill would be equal. So maybe she’d be worse at reading but better some other cognitive skill (say, special recognition). Basically there’s no free lunch—if you spend on one thing you aren’t spending it on something else.
Its not saying music is pointless. Its just saying it isnt going to help you in math. And over doing it, may in fact hurt your math due to burnout.
(Sub math for whatever non enrichment activity, sub music for any enrichment activity)
People aren’t reading the paper, just the headline, is what’s happening, and projecting their personal experience onto the 1 liner conclusion.
If you want smart kids, just give them what they need to explore, and provide the materials they need for whatever they choose to focus on.
Working on math problems might not make you a quicker thinker overall, but it will teach you math! The specific skill you enrich will become stronger at the expense of the rest of your life. I agree that time spent not socializing damages the ability to socialize (it's a joke among engineers in school). Clearly we are not making great tradeoffs, but I'm not sure this result helps us very much beyond what we already knew.
Is there some perfect version of achieving cognitive and non-cognitive skills. Are people happy if they achieve it?
Slides can be found on the lead author's homepage: http://carolinacaetano.net/
the conclusion is just one sentence and is very clear. but i was left with the feeling with all these big words that i am not supposed to understand what they are studying or their methodology...
(something that Nature did in the redesign that led to me dropping the subscription —I use "table of contents on the cover" as a sibboleth— was to introduce a section of summaries, written for a wider audience, for some of the articles, which I found, on the contrary, to be a very positive move.)
After reading through the paper, I'd like to say the authors did an excellent job addressing a sensitive subject. Part of me likes to think that they, like many of you, thought that enrichment programs were a given positive addition to life. The authors do an excellent job attempting to use mathematics to model human behavior - Sections 2-4 are almost entirely used to explain what they mean and how their equations operate. While reading the paper, I thought about some of the negative comments on this thread about economists. To that, I reasoned that we use mathematics as an attempt to model the world. Physics is our attempt to model the physical world and economics is the closest thing we have to modeling human behavior. If you disagree with that sentiment, become an economist and change the domain or dual major in data science and psychology (or sociology or any other degree used to model humans).
My biggest takeaway from this paper is that additional HOMEWORK does not help in improving a student's cognitive ability. In the paper, enrichment is defined as "investments in children's skills" which 87% of their dataset consisted of homework, reading, or before/after school academic programs. Doing or adding an additional hour of these activities does not provide any benefit to the child. This does not mean that learning is bad, as class time was a separate category, it meant "doing more" didn't help. Again, this is a sensitive subject, and I can agree that the authors don't account for "bad" instruction.
I will say there are limitations to their results, which they do acknowledge. Their dataset is small, but one of the closest datasets we have to mapping long-term human activities. I'd say this limitation grows exponentially because the CDS/PSID dataset they use only addressed 3 days in an individual's life. If I am wrong please correct me, but that is what I assume they used based on the paper. PSID contains 75,000 individuals and they only sampled 4,330 children. However, if you assume a nuclear family (4 people), that means the dataset contains ~18,750 families, which means the sample is ~25%. Again, this is a limitation, but one of the only approximations we have. The other limitation is that they are using self-response surveys, which have limited reliability.
With that, what can we take from this paper? My takeaway is that people are effectively equal but different at the same time. This was addressed with their h(X) value. Using all controllable variables, some h() function exists that produces what they addressed as "ability". This is once again where I, as did the authors, recognize that it is a sensitive topic. It is also one of the additional limitations I would say to their paper - assuming ability is a fixed value.
However, as a discussion point, 'ability' would account for some ugly truths. The g-factor and IQ are methods to address cognitive ability with sensitive undertones. Likewise, it is a commonality that networking is a key factor in successful employment often mentioned on HN. This could be accentuated by their findings that additional enrichment negatively impacted non-cognitive skills. Just because you are smart does not mean you are employable. However, I acknowledge that assessing cognitive ability is a sensitive topic, ripe with misconstruction. Likewise, I think the authors do their best to NOT say your ability is determined at birth.
I would caution against using "ability" as a metric for determining worth. Doing so would encourage early testing to see if someone is "worth the time" and I imagine would be vulnerable to prejudice and confound the features of X, e.g. a low-income child is not worth investing in. Instead, I would argue that this paper doesn't fully address "potential" with their unboservable ε. The job of the individual instead is to identify and maximize their potential (X) or at the very least improve the potential of their future selves (X+1, or children). This would acknowledge the concept of "expertise", or practicing a task makes you better at said task. It still presents an uncomfortable truth that some people aren't able to achieve success in a given task, but does address improving one's self.
I'll end there for some discussion and because it is late.
From these the conclusions should not be surprising. "Reading a book" and attending a "before or after school program" may help with the non-cognitive skills, but those are only 10% or so of "enrichment" time, and the only non-cognitive skill homework (unless it has changed greatly from my day?) would significantly help with is "Demands a lot of attention."
(silly authors: enrichment isn't meant to develop skills[1], it's meant, by choice of enrichment activities, to develop the sorts of people the children think are "one of us" as young adults, to influence whom they will hang out with, whom they will date, etc.
In Brave New World this starring of sneetches is managed in infancy by hypnopaedic suggestion:
"… all wear green," said a soft but very distinct voice, beginning in the middle of a sentence, "and Delta Children wear khaki. Oh no, I don't want to play with Delta children. And Epsilons are still worse. They're too stupid to be able to read or write. Besides they wear black, which is such a beastly colour. I'm so glad I'm a Beta." There was a pause; then the voice began again. "Alpha children wear grey They work much harder than we do, because they're so frightfully clever. I'm really awfuly glad I'm a Beta[2], because I don't work so hard. And then we are much better than the Gammas and Deltas. Gammas are stupid. They all wear green, and Delta children wear khaki. Oh no, I don't want to play with Delta children. And Epsilons are still worse. They're too stupid to be able …")
[1] I like the french idea of "education" (education), as having components not only of "how to" (savoir faire), but also of "how to live with others" (savoir vivre) and even of "how to live with oneself" (savoir être).
[2] the list of "non-cognitive" skills also shows a distinct bias towards (in Brave New World terms) Beta-Gamma ideals. How many of these soft skills does someone from a high SES background need to become, say, President of the United States?