Hacker News Reader

Comment by bonoboTP | Hacker News Reader

Stepping back a bit, I feel the argument is analogous to how music is becoming more mainstream and generic, how casual gamers "corrupt" the art form of gaming, how clickbait and stupit stuff is overshadowing lovingly curated independent websites, eternal September etc.

One answer to all these propositions is that the valuable stuff is still there, you just have an additional flood of mediocre boring predictable stuff. You can still find enough indie music and games and movies to spend all your free time on.

Similarly, if you know where to look and whose papers to watch out for, you can read more interesting stuff. The existence of more bullshit makes the filtering somewhat more effort, but it's bearable.

Most papers are never deeply read anyway, outside of the reviewers. Even citations don't mean someone really read the paper in depth.

So, most medíocre, boring papers get ignored.

It's an illusion to expect that every researcher can pump out multiple really interesting non-boring papers every year. So we pretend. We write, we cite, we present, overiflate, overpromise etc. It must look like there is steady hard noble work and toil. It's like a ritual. The scientists are writing all these papers so tax payers can sleep well that their money is paid for hard work.

View on HN

I largely agree, but we do not write all these papers "so tax payers can sleep". We write these papers because the administration incentivizes us to get the highest profile publications with the lowest effort possible. In relatively creative fields (i.e., much of computer science), this can even mean to make friends in the community and co-author papers with them without being the one who does any of the hard work. (In other sciences, it means being the one who pulls in the money, as far as I understand.)

A related problem I see is that we treat all papers the same: position (opinion) papers, reviews & surveys, and "hard" results (relevant proofs, strong empirical results, software artifacts that are deployed at scale in practice etc.). As the blog post somewhat suggests, the administration should primarily ask us to summarize these results when assessing whether we are worthy of funding and faculty positions; sure, it takes more effort, but I think it is possible.

Exactly, the big issue is that paper counts have become the dominant metric. And like with all structures dominated by a single metric, it's easy to game the system.

Most professors I have worked with in experimental sciences are completely disconnected from actual research. They just optimize for hiring hordes of people to churn out tons of mediocre papers, which they don't care about. I came from theoretical CS & math, so this was really shocking.

It's really depressing and it needs to change. Academia is now in a phase similar to that of an innovative startup which has been filled in with many middle managers, and has turned into a corporate monstrosity.

"Paper count" is about as reliable a metric as "lines of code". Publish or die is a toxic mentality in academia and you are certainly not the first to have observed this, sadly.

I'm not sure if those two fields are comparable. In software engineering, at least there's usually a dollar value impact for the code you write. In research, the impact isn't as tangible and probably won't be known for years.

> In research, the impact isn't as tangible and probably won't be known for years.

This still describes a lot of software engineering.

It's pretty outdated. In research schools you need to win funding or die nowadays.

And while publications do help you, say in getting hired and promotions, your publications are dug into pretty deeply. They don't want to be tricked into hiring someone useless who can crank out fluff but can't win a grant.

This is completely true -- something that many outside of academics don't understand -- but I'm not sure it really makes the situation any better. In a lot of ways it just kicks the problems down the road.

By whatever metric you use, it's really about the checklist rather than substance. And the checklists are based on stereotypes, and get gamed.

> This is completely true -- something that many outside of academics don't understand -- but I'm not sure it really makes the situation any better.

Indeed in some ways it's worse. When the money is insufficient, the established researchers get it first. Then everyone else has to increasingly work for established researchers as postdocs and similar positions... helping to yet further boost the advantage of the established researchers. It went from senior researchers versus junior researchers, to senior researchers with a herd of postdocs working for them, versus junior researchers.

At least if it was based on publications, everyone gets co-authorship and in fact the first author counts most. So the postdoc and the PI gain the same new items on their CV's for their collaboration. For grants, credit goes entirely to the PI.

Maybe, software solutions can be applied to academia.

In software, we have "source code distributions", such as Debian, Fedora, Gentoo, Arch, Red Hat, SUSE, etc. They are collections of mostly a same free software, with minor differences, but with different policies and purpose. Some are conservative collections of proven software, others are bleeding edge, etc.

IMHO, someone should start to collect papers, which are worth reading, and keep this collection up to date, with fixes to papers (AKA as patches in software), with one paper per topic, with ready to use math models and simulations, with in software tests for models, etc.

That sounds like Wikipedia!

Wikiscience, maybe?

It's a nice idea but there are practical problems. The first is that it supposes the scientific publication is a rational and concerted process. It is not, here is an example:

I am interested in Amyotrophic Lateral Disease and I blog about research done on ALS [0].

However there are 15,000 scientific papers published each year on this topic, on average only 20 to 30 articles per year are worth spending the time to read them (one every 500) and a milestone appears only every 3/4 years (TDP-43 in 2006, C9orf72 in 2011, etc).

After 100 years there is still no clue about ALS etiology. Scientists have strong but widely different opinions such as for some it starts in the brain motor area, for other it starts at the peripheral nerve junction with muscles, for other (new trend) it starts in the gut.

And scientists happily pursue hypothesis that their colleagues are supposed to have disproved. Sometimes the same lab (I have one in mind which is perhaps the best known in ALS research) publishes articles contradicting its previous research without discussing the issue.

How to rationally blog about this domain, how to make collections that are worth reading?

[0] https://padiracinnovation.org/News/tag/als

How is this different from compsci? :)

Every year there are 15 gazillion modules published to NPM, on average only 20-30 are worth spending the time to read, and only a handful milestones worth including in said distributions.

After 100 years there is still no clue weather microservices or monoliths are the correct hypothesis and some times the same lab publish articles contradicting previous research.

Thanks for the good laugh!

In free software distribution, we have "leaders" and "maintainers". Maintainers are volunteers which are keeping up to date just few software packages. Leaders are choosing a general direction of a distro.

IMHO, something similar can be applied to science. Leaders can chose a general direction of a collection of papers, while maintainers will work on small areas, keeping them consistent and up to date with new research.

For example:

- LIGO/VIRGO discovery of gravitational waves means that the speed of light in vacuum is not constant. It's constant in "steady" vacuum only.

- Discovery of Higgs boson means that vacuum is not an empty space, because Higgs field is present everywhere.

These recent discoveries significantly changes the field of science, so a bunch of classic paper must be updated or discarded in favor of new ones.

A huge thanks, indeed this makes sense.

Its TPS reports. That what it is. And until academia gets their own Bobs to ask academia's Lumbergh, "Yeah, Academia, lemme ask yah... real quick question here... how much time wouldja say yah spend each week dealing with these papers?"

Maybe you didn't mean it, but 'gaming the system' sounds bad. While I agree the outcome is bad, I think when your boss tells you to publish 10 paper per year and you do that, that this cannot be called 'gaming the system'

> It's really depressing and it needs to change.

It has been like that since ten years and nothing has changed since then. People find out just about the same as you did but nothing changes. It such a convenient metric it will stay for much longer.

Regarding your second paragraph, I don't think this is the case in the UK at least, where academics are judged according to the REF criteria.

Reviews are especially bad. You can tell which people work at schools where citations count in their pay (and reviews are allowed in citation counts) by the ratio of reviews to original research papers they write. Some schools/govts really need to wise up.

I think of it as a kind of predatory or scavenger behavior. They basically steal the citations from the papers they are reviewing in return for a small effort in helping us with information retrieval. We might as well just cite google and elsevier. Ideally, citations should pass through them somehow and get applied to the original sources who did the hard work.

It's true that some review papers are shallow but it's pretty cynical to paint all contributions of this type with the same brush. In my experience, good review papers are more than summaries. They can connect separate lines of research for example, compare and contrast different ideas and they can re-contextaulise a body of research to give a broader picture and reveal new and interesting directions. These papers are hard to write well and take a long time. Even more so in cases where empirical work has been undertaken so as to make direct comparisons and provide reference implementations.

Even the summary papers can be useful. On particularly active problems there can be dozens of papers per year. Sorting out wheat from chaff by highlighting notable works, and pointing out trends in published research, is again a genuine contribution, and helpful for the scientific community.

Of course they are useful, as is google scholar and the search field in various journals. Or some blog that tells you whats was new in the recent conference. That doesn't mean they deserve to be cited over the real sources. They are cited as a crutch: "go here for a more detailed list of sources". The little meta-analysis they do doesn't explain most of the citations they get.

Hard disagree. We need to incentivized more curation, distillation, summarization, comparing/contrasting, systematizing, categorizing approaches along various axes, meta analyses etc., not less.

These are often very valuable, more so than yet another 1% improvement paper that never gets reproduced.

Review papers don't steal citations. These are cited from more distant literature so readers can familiarize themselves with the topic. Those wouldn't cite all the papers mentioned in the review individually.

But as a broader point, yeah, maybe something like PageRank could be used to "pass on" the citation Ina sense.

> These are often very valuable, more so than yet another 1% improvement paper that never gets reproduced.

It's not an either-or.

And one should always read and accurately describe the contents of what you cite, meaning of course you wouldn't cite everything in a review paper directly. Only that which is relevant.

Suppose we count both separately, research citations and review citations, the latter for your great review papers or even if people just want to reference the lit review section of your research paper. What would happen to the incentive? I suggest that reviews would be less incentivized as people would primarily be concerned with rating researchers according to their research citations. Just like no one cares about your textbook sales, popular though it may be for introductory uses. This would imply reviews were not "honest" citations but are done to hack the metric.

People should care about textbooks. I think it's misguided to think that real scientific value lies only in original research.

Well you guys are pulling me to an extreme position here. Of course I love good textbooks too. And review journals like signal processing magazine are my favorites. I'm sure many people do them for great reasons too. Let's not forget that academics teach too, and there are academics and colleges that are entirely devoted to teaching, not research. People are free to prefer them more if they like. However, when it comes to research metrics, the person who did the research deserves the credit for it. Even if we subtract the citations for obvious review papers (which is commonly done) there is still the lost citations by the original researchers.

Reviews also cause other devious problems, like journals hacking their own impact factors. And don't forget the Matthew principle (aka rich get richer) whereby only big-shot researchers can get invited to have their reviews published in high-impact journals, warping the network effects to their favor even more.

I’ve heard the term “minimum knowledge increment” used as a measuring unit for papers. Dice whatever research you have into papers that contribute the minimum necessary for publication. Dicing may only produce 1 paper, good enough yet meager.

>It's an illusion to expect that every researcher can pump out multiple really interesting non-boring papers every year. So we pretend. We write, we cite, we present, overiflate, overpromise etc. It must look like there is steady hard noble work and toil. It's like a ritual. The scientists are writing all these papers so tax payers can sleep well that their money is paid for hard work.

The flip-side to this is that there is a glut of moderately motivated scientists and PhD candidates. As a result, the demand for conferences and citations is self-sustaining to meet the expectations of millions of people wanting to enter the academic world without a clear plan beyond joining the world itself.

It's as though the English-major-to-professor cycle has also become a reality in the hard sciences.

In 1916, J.J. Thomson (as in Thomson scattering; discoverer of the electron) said it perfectly [1]:

“If you pay a man a salary for doing research, he and you will want to have something to point to at the end of the year to show that the money has not been wasted. In promising work of the highest class, however, results do not come in this regular fashion, in fact years may pass without any tangible result being obtained, and the position of the paid worker would be very embarrassing and he would naturally take to work on a lower, or at any rate a different plane where he could be sure of getting year by year tangible results which would justify his salary. The position is this: You want one kind of research, but, if you pay a man to do it, it will drive him to research of a different kind. The only thing to do is to pay him for doing something else and give him enough leisure to do research for the love of it."

[1] https://archive.org/details/b29932208 - see pages 198-200, ref HN comment: https://news.ycombinator.com/item?id=21388715

I was thinking more along the lines of people who are not suited to do research in the first place. This will either be those who join the academic world out of inertia following their educational path or class/cultural pressure, or those who dislike the general labor market and want to enter the academic world due to its particularities.

On a speculative note: I believe it's yet another argument in favor of UBI

I'd assign scientists a "pension" every time they make a worthy discovery. That pension would be a monthly payment of a fixed dollar amount, it would be inalienable and paid for the rest of the scientist's life regardless of other achievements. Publishing a minor, but worthy, paper would give a 100 usd/month grant. Getting a Nobel prize would give, say, 1 million a year. However the deflation would gradually erase value of the grants.

How would that fix the actual problem? And how do you judge what is a worthy discovery and what isn't?

As parent said, UBI would probably fix the incentive scheme, your solution wouldn't. You need to decouple the reward from the result so that research is done out of sheer curiosity and love of science.

You cant decouple this (fully) with UBI. Only a little research is done at a desk, a lot need equipment, sometimes very expensive. With UBI you just get the time but not the stuff.

This was the original intention of patents. Let market value decide if the discovery was worthy and let inventor earn from license of it. We all know that's not how it works today though thanks to sloppy patent office and abusing patent trolls.

I kinda would suggest the problem in some ways is the opposite, that even people who are well-suited to what academics should be are being driven out.

It's such a mess at the moment that it's not really possible to continue out of inertia. Even if you're well-suited to research and discovery at some level, and motivated, it's a soul-crushing experience because at every step you're incentivized toward something else.

Here is a 1994 essay (also given as testimony to the US Congress) on part of of why it is a mess, from Dr. David Goostein (then vice-provost at Caltech): http://www.its.caltech.edu/~dg/crunch_art.html

In that essay, Prof. Goodstein explains how academic had been growing exponentially for decades since WWII, with each PhD giving birth to another litter of fifteen PhDs (my phrasing...), until finally that exponential process began to hit limits. That set of changes from the ending of the exponential growth in academia beginning around the 1970, leading to a break down of peer review and lots of other issues.

That's also part of why those of us (like me) who got honest well-meant advice from extremely successful academics in the 1980s or later about the value of a PhD and a career in academia were misled, even though that advice had worked for the advice giver. Freeman Dyson, by contrast, tried to steer bright people (including me) away from PhDs even in the 1980s -- having an intuition for this sort of thing. Wish I had listened to him more. :-)

Also related by Philip Greenspun from 2006 (later updated), on "Women in Science" but which also applies more broadly and is a good description of the consequences of the situation Goodstein described in 1994: http://philip.greenspun.com/careers/women-in-science "This is how things are likely to go for the smartest kid you sat next to in college. He got into Stanford for graduate school. He got a postdoc at MIT. His experiment worked out and he was therefore fortunate to land a job at University of California, Irvine. But at the end of the day, his research wasn't quite interesting or topical enough that the university wanted to commit to paying him a salary for the rest of his life. He is now 44 years old, with a family to feed, and looking for job with a "second rate has-been" label on his forehead. Why then, does anyone think that science is a sufficiently good career that people should debate who is privileged enough to work at it? Sample bias. ... A good career is one that pays well, in which you have a broad choice of full-time and part-time jobs, in which there is some sort of barrier to entry so that you won't have to compete with a lot of other applicants, in which there are good jobs in every part of the country and internationally, and in which you can enjoy job security in middle age and not be driven out by young people willing to work 100 hours per week. How closely does academic science match these criteria? I took a 17-year-old Argentine girl on a tour of the M.I.T. campus. She had no idea what she wanted to do with her life, so maybe this was a good time to show her the possibilities in female nerddom. While walking around, we ran into a woman who recently completed a Ph.D. in Aero/Astro, probably the most rigorous engineering department at MIT. What did the woman engineer say to the 17-year-old? "I'm not sure if I'll be able to get any job at all. There are only about 10 universities that hire people in my area and the last one to have a job opening had more than 800 applicants." And that's engineering, which, thanks to its reputation for dullness and the demand from industrial employers, has a lot less competition for jobs than in science. What about personal experience? The women that I know who have the IQ, education, and drive to make it as professors at top schools are, by and large, working as professionals and making 2.5-5X what a university professor makes and they do not subject themselves to the risk of being fired. With their extra income, they invest in child care resources and help around the house so that they are able to have kids while continuing to ascend in their careers. The women I know who are university professors, by and large, are unmarried and childless. By the time they get tenure, they are on the verge of infertility. ... I've taught a fair number of women students in electrical engineering and computer science classes over the years. I can give you a list of the ones who had the best heads on their shoulders and were the most thoughtful about planning out the rest of their lives. Their names are on files in my "medical school recommendations" directory. ..."

Anyway, that's all part of why I support a UBI. Give everyone enough money to live as a perpetual graduate student (without publishing and without becoming a "Disciplined Mind" as the name of a book by Jeff Schmidt on academic perils) and some of them will, which will eventually remake academia in a healthier way (including taking more chances on small-scale but far reaching basic research, whether towards cold fusion, quantum teleportation, new computer languages, new materials, new batteries, new ways of thinking, new communications patterns, new ways of conflict resolution, or whatever). If for that reason alone the USA should invest in a Universal Basic Income.

I basically mirror Thomson's opinion but there is are other frames on the thinking. Eg, as paying someone to demonstrate a deep understanding of a topic and articulate the uncertainties. That is different from traditional research because they aren't being asked to do anything exactly new. But in practice that is what I think the better researchers actually do.

Maybe there is a clue in the name "re-search". Epistemologically I don't think it is possible to set out to discover something that isn't already known to exist, so the justification for paying people would best be for evidence that they were searching rather than if they find something.

This is a great quote. Thinking of Einstein working at the patent office...

Perhaps it varies by specialty/region, but I have a good friend in the sciences here in Australia; funding has dried right up.

This year, less than 10% of NHMRC grant applications got approved, and of the ones approved most spent nearly as much time on grant applications as they did on the real work. Anyone who is only moderately motivated has retrained, because working for low pay half of the year is not appealing to anyone who can pass any doctorate program.

Even if you took the position that only the good ones are getting funded, spending 50% of your 'good' researchers time is a tragic waste of human ability.

This is a bit off-topic to papers, but perhaps there's an angle.

One answer to all these propositions is that the valuable stuff is still there, you just have an additional flood of mediocre boring predictable stuff.

I don't think this is true for most things with a mass market. I think products which have efficiencies of scale - or low marginal costs of reproduction - suck up more capital, which raises certain basic levels of standards and polish and marketing, but also greatly increases capital risk and thus reduces investor risk appetite for product novelty. These mass market products starve the mid-market of attention, which makes the mid-market less capital efficient, which means the mid-market needs to get cheaper and cheaper, lose polish, until it's less attractive to the mass market of consumers, and so on in a spiral until a certain spartan rawness becomes its own aesthetic - indie games, indie films, indie music.

I think there's a concrete example which demonstrates the actual absence of that long tail of interesting content, and not just that it's harder to find: blogs.

Blogs were great in the mid 2000s. The market did two things: it matured - the best blogs got more mindshare and turned into something closer to media organizations, much more capital intensive with full time paid writers, while and Facebook and Twitter sucked up the mass market of newsfeed consumers, both on the producer and consumer side, making it easier to produce low-value inanities and easier to consume tidbits from celebrities. This sucked the air of attention from blogs; being long-form, they were too hard to write and too long to read; and being unpaid, they didn't have slick editors or snazzy design. So a lot of them shrivelled up. There are still niche blogs, plenty of them, but a lot fewer - and we can track this loss, because we have (or had) them in our newsreaders.

To explain blogs you also have to factor in the moves toward more image and video centric content, I think.

Video content could be in itself another argument in favor of GP's thesis. There seems to be dearth of high quality video material. You have big-budget high quality videos, big-budget garbage, and then lots and lots of low-budget garbage: i.e. all the stuff "influencers" and "YouTube personas" create to live off ad revenue.

There is a small and somewhat obscure but relatively stable middle - low-budget, high-quality videos. I attribute its survival to two things: some live off Patreon donations and occasional sponsorship deals, others live because the creators have an actual dayjob, and they do YouTube as a hobby. This is similar to blogs - best ones are done on the side, with no expectation of revenue.

I see more of your latter paragraph. In several of my interest areas there is a good amount of useful content in videos. Best example is probably electronics, where a lot of tribal knowledge is now more accessible than ever before. Practical mechanics, both old fashioned (metal and woodworking) and new (CAD design, 3d-printing), also has a lot of useful video content. While there is a lot of pop-commercial "personas" making noise with their videos also, it does not seem to have hurt output by "in it for the passion" people.

The problem with research is similar but even deeper. Researchers are supposed to show feasibility or existence of some result, then applied scientists and engineers take this off into industry or elsewhere to build on it.

But there's no direct trading going on at that state to ensure they deliver what is claimed, the research "product" is just dumped into papers. And those papers serve as an impediment to subsequent researchers, since you can't publish or get funded for the same thing that's already been done and supposedly proven by someone else. Of course you wouldn't have to if they had really solved the problem. But they just threw together some minimal effort to stick their name on it first before moving on to the next low-hanging fruit somewhere else. Doing a literature review can be really frustrating.

I think I agree with a large chunk of this: there are inevitably triage/discovery problems as the number of things/options go up.

The cost of evaluating each remains fairly constant, and at some point production outstrips the ability of even extremely-interested/dedicated individuals to have a good handle on it all. Beyond this point, you either need organized collective effort (and trust) to efficiently distribute the work of keeping up (and synthesize the results) or the larger community will be wasting ever-increasing slices of potential energy to extract increasingly less knowledge of the scope of activity.

But there's a difference between simple volume growth for good reasons (i.e., people chasing incentives that are aligned at multiple levels of society), and volume growth caused by people chasing incentives that are misaligned at one or more points up the stack.

If the researchers are spending their time on things even they find boring because Goodhart's law, I think it is important to recognize it and try to undo the misaligned incentives. They're not only externalizing a cost on their entire field of knowledge--but we're also collectively suffering some ill-defined opportunity cost of whatever it was they would've spent their time on if they were following curiosity.

Inevitable of not, this has real costs, in that that government will eventually turn off the gravy train and the "good" and "bad" research alike will be negatively impacted.

In other words, academia is sowing the seeds of its own demise, and with it, the official institutional pinnacle of our culture [please don't wince too hard when reading that!].

----

The solution i think is less research but more library work. See https://blog.khinsen.net/posts/2020/07/08/the-landscapes-of-... and https://hapgood.us/2015/10/17/the-garden-and-the-stream-a-te... which it cites.

The amount of work library work we are not doing is just staggering. Consider these claims:

- There should be an accredited-author-ornly "kernel wikipedia" which the main one can choose to incorporate, almost like release vs staging branches.

- Different subfield way wish to keep their own intentionally biased corpuses, like https://ncatlab.org/nlab/

- After 6 months, virtually no one should have to read the original research article, because the librarian core will have incorporated it's results claims (however controversial) into the appropriate articles

- Textbooks / "knowledge bootstrap plans" should be continuously updated from the encyclopedia, not unlike how bootstrapping is managed in a package repo

- Librarians do the work, but also adjudicate disputes, as researchers will be naturally incentivized to contest how their work is incorporated as that is the primary way it is consumed.

> In other words, academia is sowing the seeds of its own demise

This is first order thinking. Yes, if an increasingly large quantity of academic research is becoming bullshit, eventually the ratio of good to bad research will reach an inflection point where policy will be set that reduces research funding.

This reduction in funding will have many effects, but the largest will be that as funding becomes a scarce resource, a higher curation of both researchers and topics will come into effect. There will be less research and papers overall, but the largest decrease in research will be those that are bullshit.

Finally, as more and more research will be fruitful and useful, and fewer and fewer bullshit papers are published, an inflection point will be reached. There will be more funding allocated to these exciting new areas of study, and more research will happen. Of course, with funding no longer a scarce resource, less curation will occur, and a few more bullshit papers will be published... And the cycle will have advanced one wavelength.

It's been happening for years, centuries, as a basic economic cycle, and looks something like this: _/T\_/T\_/T\

I'm thinking higher-order too, but I subscribe to the idea at the start of https://pedestrianobservations.com/2020/08/29/recession-and-...

> Question. In what ways can a recession be useful for forcing inefficient public-sector agencies to lay off redundant workers and reduce bloat?

> Answer. None.

And keep in mind that academia might as well be considered public sector, but the idea transcends to most large-scale/loosely-planned cost cutting for efficiency gain schemes.

Your starting point is a lot on individual virtue, and bad apples crowding out the good. I don't disagree that some researches are better and some are worse, but I think that any "cure" is going to be gamed and worse than the disease.

> but the largest will be that as funding becomes a scarce resource, a higher curation of both researchers and topics will come into effect

Do you have any evidence of this? I disagree in the strongest of terms. By my reckoning, curation, preventation, and other "forsight-drive" work is persistently undervalue by our society and economy. The nastiness that will accompany shrinking funding as research groups fight for short-term survival will only make that worse.

> but the largest decrease in research will be those that are bullshit

Why would those who are the best at gaming incentives now loose that skill?

> It's been happening for years, centuries, as a basic economic cycle, and looks something like this: _/T\_/T\_/T\

So after centuries of Byzantine decline, there is new group of lean and mean Greek Constantinopolitan bureaucrats?

After the Spain grew rich on New World bullion and then poor with a dearth of industrialization there is a new generation of hyper-efficient factories putting Germany to shame?

I dunno what history you are reading, but the good actors never outlive the broken system and get the last laugh in mine.

Old comment, but worth a reply.

> Your starting point is a lot on individual virtue, and bad apples crowding out the good. I don't disagree that some researches are better and some are worse, but I think that any "cure" is going to be gamed and worse than the disease.

I don't think that there will be any cure, just a change in available resources that changes the number of projects that get funding. Of course people will game it, but the people who dole out the cash will be more selective if there is less cash to dole.

>> but the largest will be that as funding becomes a scarce resource, a higher curation of both researchers and topics will come into effect

> Do you have any evidence of this? I disagree in the strongest of terms. By my reckoning, curation, preventation, and other "forsight-drive" work is persistently undervalue by our society and economy. The nastiness that will accompany shrinking funding as research groups fight for short-term survival will only make that worse.

There isn't any evidence of events that haven't occured yet. There is logical and historical support, as hundreds of years of economics have shown that market systems tend to optimize around the scarcest resources.

>> but the largest decrease in research will be those that are bullshit

> Why would those who are the best at gaming incentives now loose that skill?

They wouldn't lose that skill, but as there will be fewer research projects funded, more scrutiny will go into which projects get the funding. Sure, some will still game the system, but it will require more work, and more luck, so proportionally fewer will get funded.

>> It's been happening for years, centuries, as a basic economic cycle, and looks something like this: _/T\_/T\_/T\

> So after centuries of Byzantine decline, there is new group of lean and mean Greek Constantinopolitan bureaucrats?

Now you're just being silly... You're saying that the Institution of research will fall, rather than self correct? We'll just give up on scientific research and just... Hang out or something?

> I dunno what history you are reading, but the good actors never outlive the broken system and get the last laugh in mine.

It depends on the system. There have been years with drought, but people haven't given up on farming. Systems only fail when they are unsupported or unnecessary, and Institutional research is neither of those.

> There should be an accredited-author-ornly "kernel wikipedia"

This is roughly what the Citizendium project was aiming for. Sadly it seems it's not doing well these days.

https://en.wikipedia.org/wiki/Citizendium

https://en.citizendium.org/wiki/Citizendium

Exactly! The fact of the matter is Accreditation and the State go hand-in-hand. You can go full anarchism like Wikipedia (and thats' great!) but if you want something a bit more curated, a bit more "official", you need officers (by the etymology even!) and you need funding, i.e. state support.

Scholarpedia is similar as well.

I agree there is not enough library work and there is too much focus on the 4-15 page publication format. This amount is often not enough to fully flesh out an idea or to present something non-incremental. Rather, people tend to chunk up work, called salami publishing.

Now, sure doctoral theses also exist but the vast majority of research is presented in about 4-15 pages.

But I disagree that after 6 months everything should be integrated into other summary work.

Most published research is not worthy of being integrated into other work and 6 months are jót enough to decide. Most papers are forgotten and rightfully so. Publication doesn't mean it's correct or worthy of eternal remembrance. It just means that a scientist wants to share a finding with the research community. It's not established knowledge yet. Only when it is actually adopted and used successfully by the community, will it be part of established shared knowledge. After some years such key ideas do get written into textbooks.

Rick Beato rails against gridding and autotune in music over on YouTube.

My wife, a health outcomes researcher, points out that the expectation is that every little publishable bit be kicked out the door for a pharma project.

Historically, they would have dropped a single summary result at the end of the effort.

Now, she's trying to "autotune" the neutral or even negative results. Nominally in the name of "transparency".

If this is true, the need is a research Adele to come along and shame the riff-raff with sheer natural brilliance.

We all know autotune, but what's "gridding"?

I'm guessing this means quantization of a rhythm.

Correct, it's aligning played musical notes to a bar (measure) on a computer screen.

Then almost always, a clone tool is used to repeat that one bar identically dozens of times.

So you lose most of the organic nature of music in a mechanical fashion.

Who needs humanity when you can have mathematical perfection?

Beside humans?

I agree you're losing something that might be valuable, but it's not really part of the music, rather part of the performance. I think there's room for both though, just as we've had studio vs live recordings for the last 50 years or so.

> One answer to all these propositions is that the valuable stuff is still there

I don't understand the popularity of this argument. Whilst someone might (theoretically) have written the Great American Novel in his or her basement unbeknownst to pretty much anyone, a lot of art is fairly resource intensive and thus you can say with extremely high confidence that's its not being produced and the fault lies not with people not looking hard enough. Same with research and journalism.

In games at least, I feel that more good games are being produced.

Are papers getting more boring or does the reader just find less novels things to read as they read more and more? Maybe the reader is bored of the field in general. The 500th time you do anything isn’t as exciting as the 5th time.

On another note, why does everything have to be exciting? Small incremental improvements over the years is what I aim for.

Music is becoming more generic because musicians can rapidly reach a wide audience and iterate quickly, so they see what works and what doesn't and continue down the path of sound that produces the most money.

Isn’t the premise that this analogy extends to academia by “seeing what works and what doesn’t” to produce the most publications to the point where the majority of papers are derivative? While the iteration of the individual may not be quicker, the large increase in the number of publications seems to be indicative of the field iterating quicker.

E.g., the artist takes a big risk to produce a sound that diverges from “what works” just like a scientist takes a big risk to diverge from well-worn theory that has produced past publication fodder.

In the line of your comment about tax payers money: one of the biggest issues IMHO is the disfunctional "market" here because most of the work is paid easily by the tax payer . Particularly publishers are mostly not paying for reviews but are paid for journal and conferences largely on quantity (with few noble exceptions). The effect for a long time now was an ever increasing number of conferences (peer reviewed proceedings still make a large share in CS) and journals. What is actually worse IMHO, the review request get more and more and are largely boring. So you really have to attract the attention of the reviewer to not trigger the reflex of looking for reasons to reject (or worse the opposite, which may account to the same because there are always other reviewers, so its a matter of luck who gets assigned)

I think better structures are needed that actually incentivise good reviews. This would lead to better papers, too. There are multiple options for this. Maybe it is simply time to see reviews as publications themselves and say goodbye to flawed blinded review process. Another thing would be to take the publishers out of the game. Even IEEE and ACM have become largely a playing field for power politics between the US, China and Europe . I think we could do better and be more inclusive here. Publishing reviews and rejected work would further make meta studies much more reliable removing also bias towards positive results (who actually wants to hear about easily replicatable negative results at a conference)

The problem isn’t the existence of the papers, it’s the years of effort that go into producing the papers.

It’s the failure to harness the hard work and brainpower of so many brilliant people towards efforts that would meaningfully improve the lives of others.

Or even more reductionist there are only a finite amount of talented and smart people ever in history doing truely new and interesting stuff.

It’s just the nature of the game. You can’t just turn a dial and add more talent.

Although a bit off topic there are some authoritarian nation states who tried to do this by social engineering. Soviet Union made being an engineer a common place job title and China is trying to make tons of super smart kids at STEM. But we all know there is more to being smart than forcing kids to go down certain academic route and get top end scores (you know the whole creative side and capacity for original works to be created).

The democratic approach still seems to be the winner (not just due to brain). It’s as I mention always going to be a small minority who do great things. That doesn’t scale up artificially.

> You can’t just turn a dial and add more talent.

I'm not sure I agree there. It's notoriously hard to get/keep a research position, especially a) in academia and b) outside CS. There are a decent number of graduate student slots, some postdocs, and then you're thrown into the thunder dome.

The competition itself--and the resulting uncertainty--chases a lot of people into other careers. The people in my grad school cohort who "left" research were just as smart and often just as successful (in terms of papers, etc) as those that stayed.

It's not hard to imagine some policy tweaks that could have kept some of them in research.

>The people in my grad school cohort who "left" research were just as smart and often just as successful (in terms of papers, etc) as those that stayed.

There's research to support that actually:

https://www.insidehighered.com/news/2018/12/11/new-study-say...

Buried in that paper they basically show the hazard rate for exiting academics is sort of uncorrelated with metrics like citation rate etc.

> Or even more reductionist there are only a finite amount of talented and smart people ever in history doing truely new and interesting stuff. It’s just the nature of the game. You can’t just turn a dial and add more talent.

It’s true that we can’t turn a dial, but there must be a vast amount of human potential that goes unused every day, because people are born in impoverished countries, or suffer discrimination due to gender or race or something else, get sick with a preventable disease, or whatever other thing denies them the opportunity to make the most of themselves.

This is probably actually the norm in the world, not the exception.

So I think there’s rather a lot we could do as a species (or within individual countries) to give people opportunities and make sure potential for progress is realised

Granting your premise (which I don't personally think is accurate): currently every country in the world squanders a massive amount of talent by not providing opportunity, even starting at the most basic level of universal food security and healthcare. You may not be able to infinitely turn a dial and add more talent (or productive research/innovation), but that doesn't mean our many dials are anywhere near their maximum levels either.

Another closely related possibility is that a lot of processes in science are inherently serial, i.e. they can't scale with the number of researchers, because they require slowly integrating information and making consecutive steps of progress.

Maybe after say, quantum computers are available, it takes around 25 years for a team of researchers to slowly make progress and integrate information until they find a killer application. Funding 25 teams of researchers may make them pump out papers 25x faster, but they aren't going to find the killer application in 1 year.

> So we pretend. We write, we cite, we present, overiflate, overpromise etc.

> so tax payers can sleep well that their money is paid for hard work.

It's a bit contradictory.

I won't go into scandalized rant because it's useless, and there's no point shouting against the universe from a bedroom chair. Still it's a bit disheartening. I wish the scientific world could reinvigorate and pump some spark back into the field.

"Science advances one funeral at a time"

-- paraphrased from Max Planck

All what you say is true and I appreciate your deep thoughts.

There is another dimension to that though. It's the long tail. The internet has tremendously facilitated mixing existing content. That's creative too but not in the original sense of creating something new.

With detached knowledge before the internet age much less content has been produced and I would assume the novelty to mixed content ratio was very much in favour of new content instead of mixed content.

Internet technology brought us search engines but their search algorithmus favour those paying most instead of providing a service to discriminated remixed content from new content.

Reply on news.ycombinator.com