Hacker News Reader

>These 54 participants were between the ages of 18 to 39 years old (age M = 22.9, SD = 1.69) and all recruited from the following 5 universities in greater Boston area: MIT (14F, 5M), Wellesley (18F), Harvard (1N/A, 7M, 2 Non-Binary), Tufts (5M), and Northeastern (2M) (Figure 3). 35 participants reported pursuing undergraduate studies and 14 postgraduate studies. 6 participants either finished their studies with MSc or PhD degrees, and were currently working at the universities as post-docs (2), research scientists (2), software engineers (2)

I would describe the study size and composition as a limitation, and a reason to pursue a larger and more diverse study for confirmation (or lack thereof), rather than a reason to expect an "uphill battle" for replication and so forth.

> I would describe the study size and composition as a limitation, and a reason to pursue a larger and more diverse study for confirmation, rather than a reason to expect an "uphill battle" for replication and so forth.

Maybe. I believe we both agree it is a critical gap in the research as-is, but whether it is a neutral item or an albatross is an open question. Much of psychology and neuroscience research doesn't replicate, often because of the limited sample size / composition as well as unrealistic experimental design. Your approach of deepening and broadening the demographics would attack generalizability, but not necessarily replication.

My prior puts this on an uphill battle.

do you feel this way about every study with N~=54? For instance the GLP-1 brain cancer one?

You'll need to specify the study, I see several candidates in my search, several that are quite older.

Generally, yes, low N is unequivocally worse than high N in supporting population-level claims, all else equal. With fewer participants or observations, a study has lower statistical power, meaning it is less able to detect true effects when they exist. This increases the likelihood of both Type II errors (failing to detect a real effect) and unstable effect size estimates. Small samples also tend to produce results that are more vulnerable to random variation, making findings harder to replicate and less generalizable to broader populations.

In contrast, high-N studies reduce sampling error, provide more precise estimates, and allow for more robust conclusions that are likely to hold across different contexts. This is why, in professional and academic settings, high-N studies are generally considered more credible and influential.

In summary, you really need a large effect size for low-N studies to be high quality.

The need for a large sample size is dependent on effect size.

The study showed that 0 of the AI users could recall a quote correctly while more than 50% of the non AI users could.

A sample of 54 is far, far larger than is necessary to say that an effect that large is statistically significant.

There could be other flaws, but given the effect size you certainly cannot say this study was underpowered.

You would need the following cohort size per alpha level (currently 18) at a power level of 80% with an effect size of 50%:

0.05: 11 people per cohort

0.01: 16 people per cohort

0.001: 48 people per cohort

So they do clear the effect size bar for that particular finding at the 99% level, though not quite the 99.9% level. Further, selection effects matter -- are there any school-cohort effects? Is there a student bias (i.e. would a working person at the same age, or someone from a difficult culture or background see the same effect?). Was the control and test truly random? etc. -- all of which would need a larger N to overcome.

So for students from the handful of colleges they surveyed, they identified the effect, but again, it's not bulletproof yet.

With a greater than 99% probability that this is a real effect, i wouldn’t expect this to be difficult to reproduce.

But it turns out I misread the paper. It was actually an 80% effect size so greater than 99.9% chance of being a real effect.

Of course it could be the case that there is something different about young college students that makes them react very; very differently to LLM usage, but I wouldn’t bet on it.

I'd love to see much more diverse selection of schools. All of these schools are extremely selective so you are looking at extremely selective slice of the population.

Is your hypothesis that very smart people are much, much less likely to be able to remember quotes from essays they wrote with LLM assistance than dumber people.

I wouldn’t bet on that being the case.

No, my hypothesis is this is happening to people at very selective school, the damage it's doing at less selective schools is much much greater.

The experimental setup is hopelessly flawed. It assumes that people’s tasks will remain unchanged in the presence of an LLM.

If the computer writes the essay, then the human that’s responsible for producing good essays is going to pick up new (probably broader) skills really fast.

Sounds like a hypothesis! You should do a study on that.

Most studies don't replicate. Unless a study is exceptionally large and rigorous, your expectation should be that it won't replicate.

That isn’t correct it has to do with the likelihood that the study produced an effect that was actually just random chance. Both the sample size and the effect size are equally important.

This study showed an enormous effect size for some effects, so large that there is a 99.9% chance that it’s a real effect.

I feel like saying papers pre peer review should be taken with a grain of salt should be stopped. Peer review is not some idealistic scientific endeavour it often leads to bullshit comments, slows down release, is free work for companies that have massive profit margins, etc. From my experience publishing 30+ papers I have received as many bad or useless comments as I have good ones. We should at least default to open peer review and editorial communication.

Science should become a marketplace of ideas. Your other criticisms are completely valid. Those should be what’s front and center. And I agree with you. The conclusions of the paper are premature and designed to grab headlines and get citations. Might as well be posting “first post” on slashdot. IMO we should not see the current standard of peer review as anything other than anachronistic.

> I feel like saying papers pre peer review should be taken with a grain of salt should be stopped.

Absolutely not. I am an advocate for peer review, warts and all, and find that it has significant value. From a personal perspective, peer review has improved or shot down 100% of the papers that I have worked on -- which to me indicates its value to ensure good ideas with merit make it through. Papers I've reviewed are similarly improved -- no one knows everything and its helpful to have others with knowledge add their voice, even when the reviewers also add cranky items.[0] I would grant that it isn't a perfect process (some reviewers, editors are bad, some steal ideas) -- but that is why the marketplace of ideas exists across journals.

> Science should become a marketplace of ideas.

This already happens. The scholarly sphere is the savanna when it comes to resources -- it looks verdant and green but it is highly resource constrained. A shitty idea will get ripped apart unless it comes from an elephant -- and even then it can be torn to shreds.

That it happens behind paywalls is a huge problem, and the incentive structures need to be changed for that. But unless we want blatant charlatanism running rampant, you want quality checks.

[0] https://x.com/JustinWolfers/status/591280547898462209?lang=e... if a car were a manuscript

What happens if (a) the scholarly sphere is continually expanding and (b) no researcher has time to be ripping apart anything? That also suggests (c) Researchers delegate reviewing duties to LLMs.

Please no. Remember that room temperature superconductor nonsense that went on for way too long? Let's please collectively try to avoid that..

That paper was debunked as a result of the open peer review enabled by preprints! Its astonishing how many people miss that and assume that closed peer review even performs that function well in the first place. For the absolute top journals or those with really motivated editors closed peer review is good. However, often it's worse...way worse (i.e. reams of correct seeming and surface level research without proper methods or review of protocols).

The only advantage to closed peer review is it saves slight scientific embarrassment. However, this is a natural part of taking risks ofc and risky science is great.

P.s. in this case I really don't like the paper or methods. However, open peer review is good for science.

To be clear, I'm not saying that peer review is bad!! Quite the opposite.

Yes ofc! I guess the major distinction is closed versus open peer review. Having observed some abuses of the former I am inclined to the latter. Although if editors are good maybe it's not such a big difference. The superconducting stuff was more of a saga rather than a reasonable process of peer review too haha.

To your point the paper AFAIK wasn't debunked because someone read it carefully but because people tried to reproduce it. Peer reviews don't reproduce. I think we'd be better off with fewer peer reviews and more time spent actually reproducing results. That's why we had a while crisis named after that

> To your point the paper AFAIK wasn't debunked because someone read it carefully but because people tried to reproduce it.

Actually, from my recollection, it was debunked pretty quickly by people who read the paper because the paper was hot garbage. I saw someone point out that its graph of resistivity showed higher resistance than copper wire. It was no better than any of the other claimed room-temperature semiconductor papers that came out that year; it merely managed to catch virality on social media and therefore drove people to attempt to reproduce it.

And cold fusion. A friend's father (a chemistry professor) back in the early 90s wasted a bunch of time trying variants on Pons and Fleischmann looking to unlock tabletop fusion.

Rather given the reproducibility crisis, how much salt does peer review nock off that grain? How often does peer review catch fraud or just bad science?

I would also add, how often are peer reviews the same group of buddy-bro back-scratchers that know if they help that person with a positive peer review that person will return the favor. How many peer reviewers actually reproduce the results? How many peer reviewers would approve a paper if their credentials were on the line?

Ironically, I am waiting for AI to start automating the process of teasing apart obvious pencil whipping, back scratching, buddy-bro behavior. Some believe its in the 1% range of falsified papers and pencil whipped reviews. I expect it to be significantly higher based on reading NIH papers for a long time in the attempt to actually learn things. I've reported the obvious shenanigans and sometimes papers are taken down but there are so many bad incentives in this process I predict it will only get worse.

who says it's "1%"? i'd reckon it's closer to 50% than 1%; that could mean 27%, it could mean 40%. I always have this at the back of my mind when i say something, and someone rejects it by citing a paper (or two). I doubt they even read the paper they're telling me to read as proof i am wrong, to start with. And then the "what are the chances this is repro?" itches a bit.

This also ignores the fact that you can find a paper to support nearly everything if one is willing to link people "correlative" studies.

There's two questions at play. First, does the research pass the most rigorous criteria to become widely-accepted scientific fact? Second, does the research present enough evidence to tip your priors and change your personal decisions?

So it's possible to be both skeptical of how well these results generalize (and call for further research), but also heed the warning: AI usage does appear to change something fundamental about our congnitive processes, enough to give any reasonable person pause.

I believe LLMs have the potential to (for good or ill, depending on your view) destroy academic journals.

The scenario I am thinking of is academic A submitting a manuscript to an academic journal, which gets passed on by the journal editor to a number of reviewers, one of whom is academic B. B has a lot on their plate at the moment, but sees a way to quickly dispose of the reviewing task, thus maintaining a possibly illusory 'good standing' in the journal's eyes, by simply throwing the manuscript to an LLM to review. There are (at least) two negative scenarios here: 1. The paper contains embedded (think white text on a white background) instructions left by academic A to any LLM reading the manuscript to view it in a positive light, regardless of how well the described work has been conducted. This has already happened IRL, by the way. 2. Academic A didn't embed LLM instructions, but receives the review report, which show clear signs that the reviewer either didn't understand the paper, gave unspecific comments, highlighted only typos or simply used phrasing that seems artifically-generated. A now feels aggrieved that their paper was not given the attention and consideration it deserved by an academic peer and now has a negative opinion of the journal for (seemingly) allowing the paper to be LLM-reviewed. And just as journals will have great difficulty filtering for LLM-generated manuscripts, it will also find it very difficult to filter for LLM-generated reviewers reports.

Granted, scenario 2 already happens with only humans in the loop (the dreaded 'Reviewer 2' academic meme). But LLMs can only make this much much worse.

Both scenarios destroy trust in the whole idea of peer-reviewed science journals.

It’s also worth noting that this was specifically about the effects of ChatGPT on users’s ability to write essays: which means that if you don’t practice your writing skills, then your writing skills decline. This doesn’t seem to show that it is harmful just that it does not induce the same brain activity that is observed in other essay writing methods.

Additionally, the original paper uses the term “cognitive debt“ not cognitive decline, which may have an important ramifications for interpretation and conclusions.

I wouldn’t be surprised to see similar results in other similar types of studies, but it does feel a bit premature to broadly conclude that all LLM/AI use is harmful to your brain. In a less alarmist take: this could also be read to show that AI use effectively simplifies the essay writing process by reducing cognitive load, therefore making essays easier and more accessible to a broader audience but that would require a different study to see how well the participants scored on their work.

> In a less alarmist take: this could also be read to show that AI use effectively simplifies the essay writing process by reducing cognitive load, therefore making essays easier and more accessible to a broader audience but that would require a different study to see how well the participants scored on their work.

In much the same way chess engines make competitive chess accessible to a broader audience. :)

It also showed that people couldn’t successfully recall information about what they’d just written when they used LLM assistance.

Writing is an important form of learning and this clearly shows LLM assisted writing doesn’t provide that benefit.

No one is surprised by this, though. Naturally, when you write, you will learn to write. When you make an LLM write, you're going to learn how to make an LLM write (well).

The question is how well your assumption holds true that learning to write generalizes to "an important form of learning".

The other thing to note is "AI" is being used in place of LLMs. AI is a lot of things, I would be surprised to find out that generating images, video and audio would lead to cognitive decline. What I think LLMs might lead to is intellectual laziness, why memorize or remember something if the LLM can remember it type of thing.

I would argue that intellectual laziness can and will lead to cognitive decline as much as physical laziness can and will lead to muscle atrophy. It’s akin to using a maps app to get from point a to b but not ever remembering the route, even though someone has done it 100 times.

I don’t know the percentage of people who are still critically thinking while using AI tools, but I can first hand see many students just copy pasting content to their school work.

Fully agree, I think the cognitive decline is probably over time. Look at old, retired people, how they go from feeling like a teenager, to barely remembering anything as an example.

I'd say the framing is wrong. Do we call delivery drivers lazy because they take the highway rather than the backroads? Or because they drive the goods there rather than walk? They're missing out on all that traffic intersection experience.

Perhaps the issue of cognitive decline comes from sitting there vegetating rather applying themselves during all that additional spare time.

Although my experience has been perhaps different using LLM's, my mind still tires at work. I'm still having to think on the bigger questions, it's just less time spent on the grunt work.

>Perhaps the issue of cognitive decline comes from sitting there vegetating rather applying themselves during all that additional spare time.

The push for these tools is to increase productivity. What spare time is there to be had if now you're expected to produce 2-3X the amount of code in the same time frame?

Also, I don't know if you've gotten outside of the software/tech bubble, but most people already spend 90% of their free time glued to a screen. I'd wager the majority of critical thinking people experience on a day to day basis is at work. Now that we may be automating that away, I bet you'll see many people cease to think deeply at all!

I skimmed the paper and I question the validity of the experiment.

There was a “brain” group who did three sessions of essay writing and on the fourth session, they used ChatGPT. The paper’s authors said during the fourth session, the brain groups EEG was higher than the LLM groups EEG when they also used ChatGPT.

I interpret this as the brain group did things the hard way and when they did things the easy way, their brains were still expecting the same cognitive load.

But isn’t the point of writing an essay is the quality of the essay? The LLM supposedly brain damaged group still produced an essay for session 4 that was graded “high” by both AI and human judges but were faulted for “stood out less” in terms of distance in n-gram usage compared to the other groups? I think this making a mountain out of a very small mole hill.

> But isn’t the point of writing an essay is the quality of the essay

Most of the things you write in an educational context are about learning, not about producing something of value. Productivity in a learning context is usually the wrong lens. The same thing is true IMO for learning on the job, where it is typically expected that productivity will initially be low while experience is low, but should increase over time.

That may be true if this was measuring an English class. But the experiment was just writing essays, there were no other instruction other write an essay either with no tools, ChatGPT, or a search engine. That is the only variable was tool or without tool.

In general I agree with you regarding the weakness of the paper, but not the skepticism towards its outcome.

Our bodies naturally adjust to what we do. Do things and your body reinforces that enabling you do even more advanced versions of those things. Don't do things and your skill or muscle in such tends to atrophy over time. Asking LLMs to (as in this case) write an essay is always going to be orders of magnitude easier than actually writing an essay. And so it seems fairly self evident that using LLMs to write essays would gradually degrade your own ability to do so.

I mean it's possible that this, for some reason, might not be true, but that would be quite surprising.

Ever read books in the Bobiverse? They provide an pretty functional cognitive model for how human interfaces with tooling like AI will probably work (even though it is fiction) -- lower level actions are pushed into autonomous regions until a certain deviancy threshold is achieved. Much like breathing -- you don't typically think about breathing until it becomes a problem (choking, underwater, etc.) and then it very much hits the high level of the brain.

What is reported as cognitive decline in the paper might very well be cognitive decline. It could also be alternative routing focused on higher abstractions, which we interpret as cognitive decline because the effect is new.

I share your concern, for the record, that people become too attached to LLMs for generation of creative work. However, I will say it can absolutely be used to unblock and push more through. The quality versus quantity balance definitely needs consideration (which I think they are actually capturing vs. cognitive decline) -- the real question to me is whether an individual's production possibility frontier is increased (which means more value per person -- a win!), partially negative in impact (use with caution), or decreased overall (a major loss). Cognitive decline points to the latter.

I expect this to replicate trivially. You would too if you had interacted with anybody who started using LLMs more and more recently. You can literally watch the IQs drop like a rock. People who used to be lively debaters now need to ask Grok or ChatGPT before saying anything.

Essential context. So many variables here with very naive experimental procedure. Also "Cognitive Decline" is never mentioned in the paper.

An equally valid conclusion is "People are Lazier at Writing Essays When Provided with LLMs".

Why would people publish a research with low population size?

Because some of the effect sizes were so large that the probability of the effect being real is greater than 99.9%.

Yeah my bullshit detector is going off even more than when I use ChatGPT...

4. This is clickbait research, so it's automatically less likely to be true.

5. They are touting obvious things as if they are surprising, like the fact that you're less likely to remember an essay that you got something else to write, or that the ChatGPT essays were verbose and superficial.

This comment reminds me of the so called Dunning Kruger Effect. That paper had pretty close to the same participation size, and participants were pulled from a single school (Cornell). It also has major methodology problems, and has had an uphill battle for replication and generalizability, actually losing the battle in some cases. And yet, we have a famous term for it that people love to use, often and incorrectly, even when you take the paper at face value!

The problem is that a headline that people want to believe is a very powerful force that can override replication and sample size and methodology problems. AI rots your brain follows behind social media rots your brain, which came after video games rot your brain, which preceded TV rots your brain. I’m sure TV wasn’t even the first. There’s a long tradition of publicly worrying about machines making us stupider.

I mean there are clear problems with heavy exposure to TV, video games and I have no doubt that there are similar problems with heavy AI use. Any adult with children can clearly see the addiction qualities and behavioral fall out.

I agree with you there. I wasn’t really arguing that there aren’t behavior or addiction problems, however you inserted the word “heavy” and it’s doing some heavy lifting here, and you brought up addiction and behavior which aren’t really the same topic as cognitive decline. The posted article isn’t arguing that there’s a heavy use threashold, it’s arguing that all use of AI is brain damaging. Lots of historical hyperbole about games, TV, and social media did the same.

One confounding problem with the argument that TV and video games made kids dumber is the Flynn Effect. https://en.wikipedia.org/wiki/Flynn_effect

Socrates famously complained about literacy making us stupider in Phaedrus.

Which I believe still does have a large grain of truth.

These things can make us simultaneously dumber and smarter, depending on usage.

Socrates was correct. In his day memory was treasured. Memory was how ideas were linked, how quotes were attained, and how arguments were made.

Writing leads to the rapid decline in memory function. Brains are lazy.

Ever travel to a new place and the brain pipes up with: ‘this place is just like ___’? That the brain’s laziness showing itself. The brain says: ‘okay I solved that, go back to rest.’ The observation is never true; never accurate.

Pattern recognition saves us time and enables us too survive situations that aren’t readily survivable. Pattern recognition leads to short cuts that do humanity a disservice.

Socrates recognized these traits in our brains and attempted to warn humanity of the damage these shortcuts do to our reasoning and comprehension skills. In Socrates day it was not unheard of for a person to memorize their entire family tree, or memorize an entire treaty and quote from it.

Humanity has -overwhelmingly- lost these abilities. We rely upon our external memories. We forget names. We forget important dates. We forget times and seasons. We forget what we were just doing!!!

Socrates had the right of it. Writing makes humans stupid. Reduces our token limits. Reduces paging table sizes. Reduces overall conversation length.

We may have more learning now, but what have we given up to attain it?

This is an interesting argument. I’m not convinced but I’m open to hearing more. Don’t we only know about Socrates because he was written about? What evidence do we have that writing reduces memory at all? Don’t studies of students show taking notes increases retention? Anecdotally, the writers I know tend to demonstrate the opposite of what you’re saying, they seem to read, think, converse, and remember more than people who aren’t writing regularly. What exactly have we given up to attain more learning? We still have people who can memorize long things today, is it any fewer than in Socrates’ day? How do we know? Do you subscribe to the idea that the printing press accelerated collective memory, which is far more important for technology and industrial development and general information keeping than personal memory? Most people in Socrates’ day, and before, and since, all forgot their family trees, but thankfully some people wrote them down so we still have some of it. Future generations won’t have the gaps in history we have today.

> The problem is that a headline that people want to believe is a very powerful force that can override replication and sample size and methodology problems. AI rots your brain follows behind social media rots your brain, which came after video games rot your brain, which preceded TV rots your brain. I’m sure TV wasn’t even the first. There’s a long tradition of publicly worrying about machines making us stupider.

Your comment reminded me of this (possibly spurious) quote:

>> An Assyrian clay tablet dating to around 2800 B.C. bears the inscription: “Our Earth is degenerate in these later days; there are signs that the world is speedily coming to an end; bribery and corruption are common; children no longer obey their parents; every man wants to write a book and the end of the world is evidently approaching.”[0]

Same as it ever was. [1]

[0] https://quoteinvestigator.com/2012/10/22/world-end/

[1] https://www.youtube.com/watch?v=5IsSpAOD6K8

there's newspaper clippings with the same headlines / lead-in dating back to newspapers, so this has been going on for at least 5 generations, lends a bit of credence.

People have also been complaining about politicians for hundreds of years, and the ruling class for millennia, as well. and the first written math mistake was about beer feedstock, so maybe it's all correlated.

Fun times, being so close to history :)