I have found this as well. However it's important to remember that before one can do good work, one has to find funding for the good work. It's a complex problem.
It's not even publishing crap. It's that these places develop into odd little societies that end up resembling Orwell's Animal Farm.
Obviously, this very good lab is collapsing. I have moved to Oxbridge now and things are much better. Sad to see so many cancer research Euros wasted.
To expand a bit...it's not that nearly everyone doesn't start out super excited about their subjects and motivated by advancing human knowledge. It's that shortly post-PhD, you realize that in order to continue doing this work you find exciting and meaningful, you have to get money (for supplies, equipment, office space, not to mention your own salary), and the funding system is inherently kind of broken. It's not just the disappearance of tenure-track jobs. To caricature somewhat:
1. Funding is limited - grant success rates in my field are currently 10% to 15% - and in order to have a chance you need some history of having done interesting work, the more the better.
2. To carry out more of your ideas (to maximize the chances that some of them will turn out to be very interesting, as well as the overall amount and speed of work) you typically need more manpower than just yourself. You may also need equipment and people to operate it. Since you're probably a bit short of money, you hire trainees (students or postdocs), who cost less. Now you are really feeling the funding pressure since their livelihoods, not only your own, depend on your getting enough funds.
3. Eventually those trainees graduate, or become senior. Now they need to apply for their own grants and you need to find new trainees. In other words, the process of science today inherently increases the number of scientists competing for funding, and because each scientist during a single career typically requires many trainees, this number increases exponentially. There is no way the research funding budget can increase that fast over the long term.
4. The constantly increasing funding pressure means more and more people become preoccupied with their own survival above most any other professional concern. In addition to politics and ladder climbing, it hurts the science directly: if some project's not likely to get future funding, you might feel you don't have time for it, even if that's what interests you most about your area.
It's a hard problem...my impression is the reforms needed would be so sweeping that I'm not sure anyone has a complete picture of what things to do instead, let alone implement them.
The proposal: the grant-giving institution instead earmark Z% of $X for endowing new tenured professor chairs at the schools.
This reduces the ratio of (postdocs + students) per professor. In the extreme case, if all institutions set Z=100%, there would be only professors and no students or postdocs; the number of professors would be less than the number of professors+postdocs+students is today, but this number would be growing at a substantially faster rate (there will be some critical value of Z, large but <100%, such that the the number of students funded is roughly equal to or slightly greater than the replacement rate of retiring professors). Note that this is not all-or-nothing; you can incrementally convince one grant-giving institution at a time to implement this reform, with Z set at various levels, and see incremental benefits.
"There is a general rejection of these alternative paths to knowledge dissemination in academia, but I have grown out of caring about it. Selling knowledge and prestige are the bread and butter of universities, so we should not be surprised to see the main recipients of the flow of money coming from well-wishing parents and governmental funding agencies dismiss the validity of other, less socially costly paths to knowledge dissemination."
Seems like he's found other more effective ways for him and for others learning than to do more of the "good work"[0].
[0] "…produce multiple scientific articles per year, none of which having any significant impact on our understanding of the world, similarly to how a chicken who's head has been cut can still walk, but not toward any sort of goal. This issue reveals itself in a series of noxious conditions that are affecting me and my colleagues: a high number of scientific articles with fraudulent data due to the pressures of the "publish or perish" system makes it impossible to know if a recent discovery is true or not; a large portion of the time of a scientist is spent just writing grants so that they can be submitted to 5-10 agencies in the hope that one of them will accept; and a scientific publication system that has become so corrupted that it is almost impossible to get a scientific article published without talking one-on-one with the editor before submitting the article."
The main issue as I see it is not that good papers are not rewarded, but rather that bad/mediocre papers are rewarded far more than they deserve. This means that the incentive to work hard on a breakthrough is diminished.
The problem is, like this article alludes to, that the field does not reward scientific curiosity or even serious scientific advancement.
If you're a smart first-worlder, there's many careers more lucrative than scientific research, so you'd only go into it because of passion. But if you're a smart third-worlder, a scientific career is often your best shot at emigrating to a rich country, so it becomes rational to do it whether you're passionate or not.
The problem is made harder by the near-extinction of tenure-track positions. Colleges are using adjuct-profressors exclusively now, because they are cheap, plentiful and hungry. The search for knowledge has turned into a mill to train people and get a paycheck.
Nearly everyone I know that graduated with a PhD from an R1 universities, and was what I considered reasonably good (they were not complete goof-offs and were serious about their work) got good tenure-track positions 0-2 years after graduating.
Would these people perhaps be clustered in a particular field? I'd guess that things are very different depending on the percentage of PhD's in a field who go into academia versus industry. For example, the academic job market for Astronomy is very different than CS, because there are many more non-academic options for CS graduates.
I think even an outstanding Astronomy PhD from a top tier university may find it difficult to get a tenure track position in their field. The same would probably be true for CS if all the graduates were to pursue academic positions, but since many go into industry, there is less competition for tenure track positions among those who want to teach.
10% of astronomers get the PhD, 10% of the PhDs get post docs, 10% of post docs get tenure.
And tenure in 0-2 years? That's impossible outside of China or India or the Soviet bloc. I think the average amount of time an Astronomer spends in post-docs is 9-12 years.
Then I read the reference and it said "To this end, the TIAA-CREF Institute surveyed full-time college and university faculty age 60 and older regarding issues related to whether and when to retire." The survey is about seniors (in terms of age) because TIAA-CREF is a retirement account company, not "senior tenure" whatever that means. I'm actually more surprised that the number of professors over 60 is not 100% since that's the only population surveyed.
The rest of the Wikipedia page is similarly biased, "Today (2015), adjunct professors, hired for their low cost only, rather than expertise, make up more than half the teaching faculty at United States universities." I can't tell if the Wikipedia article is intentionally ridiculous or if it's some parody article.