MIT president: Why so many optimistic scientists are losing heart
bostonglobe.com
bostonglobe.com
In my view, you need to have two projects going on; one that satisfies funders and pays the bills, possibly the latest buzzword. Then alongside that a more theoretical project in an area that you are passionate about. Plod along in the background with the latter to make the former easier.
Satisfying the task masters who are often very stuck in their ways and risk adverse is the hard part. Pushing the boundaries to give out a little more than they would like is essential.
Of course, if you make a statistics over a very large number of people, then you will get that on average the older people will be more conservative and less likely to come with revolutionary theories.
The problem is that such statistics are absolutely useless, because they do not provide any kind of predictive ability that could lead to correct actions in concrete cases, i.e. when discussing individuals or teams of a typical size, because the variance and the number of outliers are too great.
There are an extremely large number of older people who can come with much more revolutionary ideas than the vast majority of younger people. I have seen far less cases of young people with revolutionary ideas than old people.
One reason is that young people are still under the strong influence of what they have learned in schools or other such environments, and they are still not aware that many things among what they have learned are actually wrong. Typically only after many years of practical experience they learn to become more skeptical and to question everything that they have learned and everything that happens to be widely considered as true. Most people never reach this stage.
There are many people who learn something new every day of their life, until they might be a century old, while the majority of people, regardless of their academic qualifications and how young they may be, are reluctant to learn anything that appears to contradict what they already believe.
Therefore, anyone who encounters two unknown people, one young and one old, and who believes that the young one has more chances to create a revolutionary theory than the other, is just stupid.
This being statistics things are more nuanced and hence interpretations/conclusions can be debated but the data cannot be denied. Of course we also have to look at the suitability of the data itself.
The study that the article references is Aging and the Narrowing of Scientific Innovation - https://arxiv.org/abs/2202.04044 This is not long and well worth reading.
A related earlier study is Papers and patents are becoming less disruptive over time which was already discussed on HN - https://news.ycombinator.com/item?id=34248858 The Economist's overview - https://archive.ph/X1aU4.
The basic thesis of the paper that ageing is negatively correlated with disruptive innovation is nothing new. It has been noted by many scientists (eg. Max Planck - "Science advances one funeral at a time") and Thomas Kuhn also pointed it out in his The Structure of Scientific Revolutions. Excerpts from the the study;
Scientific careers today are marked by growing polarization. A small number of scientists now remain active and influential for longer than ever (1), while many others pass through research as temporary workers (2). Lengthened training periods (3, 4), the elimination of mandatory retirement (5), and funding systems that reward experience have concentrated resources among senior scientists (6). As science becomes increasingly dependent on its aging core, a central question arises: How does academic age influence creativity?
... Yet empirical evidence often shows the opposite: innovation tends to come from younger scientists who are less tied to convention
... In other words, experience aids recombinatorial novelty (19) but limits disruptive innovation (20), which challenges prior work and drives the creative destruction of ideas (21).
We test this pattern of intellectual aging across modern science by distinguishing between two empirically measured forms of creativity. Novelty links previously unconnected ideas, applying established knowledge in new contexts (19, 28, 29). Disruption breaks existing linkages, displacing established knowledge with new ideas (30–32). Although both drive progress, they reshape scientific knowledge in opposite ways: novelty extends established paradigms by revealing hidden complementarities between ideas, whereas disruption overturns them by exposing hidden substitutions (33).
Analyzing more than 3.6 million scientists who published between 1960 and 2020, we find that novelty increases while disruption declines with academic age. We trace this divergence to a “Nostalgia Effect”—older scientists’ tendency to cite earlier work. This preference for the past stimulates new combinations of familiar ideas, enhancing novel combinations, but it also anchors attention to established paradigms, reducing disruption and fostering criticism of new ideas proposed by younger colleagues. As scientists age, they also form more hierarchical teams, amplifying their reach yet reinforcing reliance on conventional knowledge.
Our study contributes to three areas. First, it clarifies why prior research on age and creativity has been mixed (7–12): aging enhances combinatorial innovation but limits disruptive breakthroughs. Second, it links these cognitive dynamics to team structure, explaining why larger, hierarchical teams tend to innovate less (31, 50, 51), thereby revealing the social foundations of intellectual aging. Third, it identifies the Nostalgia Effect as a measurable cognitive process that shapes how scientists remember, connect, and—at times—struggle to forget ideas over their careers (52–54).
The paper details how they measured novelty vs. disruption (debatable) and more importantly social effects of ageing on science which is most interersting.
These analyses allow us to examine the life cycle of scientific aging, which we emphasize is not merely biological but deeply social. As scientists progress from trainees to principal investigators, their roles and constraints shift markedly. Increasing leadership, administrative, and reviewing responsibilities can reshape priorities and limit the time available to stay current with the research frontier, thereby slowing reference renewal. In this sense, individual aging also carries collective consequences. Within teams, senior scientists’ preferences can shape the citation practices of their collaborators, especially those they lead. In peer review, older reviewers may direct authors toward citing familiar or preferred work, further reinforcing established literature. Together, these social mechanisms reveal how aging scientists shape the collective evolution of scientific memory.
Critically, role transitions magnify individual tendencies. As careers advance, scientists shift from producing research to leading teams and institutions, reviewing papers and grants, and mentoring students (7). These social roles shape science’s collective memory and evaluative standards, channeling attention toward established ideas and away from conceptual replacements. What appears as a decline thus also reflects functional reorientation: from overturning paradigms to consolidating knowledge and governing the conditions under which younger scientists pursue disruptive innovation.
They finally end with policy recommendations to better balance novelty vs. disruptive work in science. It is important to note that Science has always progressed with both Continuity/Consolidation/Novelty and Disruption working together and alternately. Note that it is not a value judgement on the scientists working in the two domains. Both are needed but at present time it seems scientific work is getting increasingly skewed in the direction of the former.