Time to break academic publishing's stranglehold on research
newscientist.com
newscientist.com
I'm a shareholder and board member of a large privately-held, family-owned academic publishing company. If anyone is interested in trying to understand what makes the industry work, why it's so hard to disrupt it, etc. I'd love to engage or put you in touch with people within the industry smarter than me - my email is in my profile.
I know the industry is particularly frustrating to the HN crowd. We want to think it's a technology problem - that distributing PDFs is a solved problem (which it obviously is). But the root of the problems (of which there are many) are all cultural and much harder to change. If you're going to jump in and try to "fix" the industry or put publishers out of business, I highly encourage you to engage with folks in the industry with an open mind and really try to understand why things work the way they work. You're not going to have any success unless you truly understand the incentive structure of academia and the social and cultural aspects of inertia that are at play. If you go in thinking you can build a better "publishing" mousetrap you will fail. You have to realize publishers are in the reputation business. And when you start peeling back the onion of how academics are assessed, given jobs, given tenure, etc you start seeing how hard changing behavior can be.
That's right, and that's also why the prestigious researchers and universities, those with an already established reputation, have a responsibility: collectively leave the editorial boards of for-profit publishers, set up alternative venues with the help of university libraries. Share the archival, indexing and discovery effort among universities via peer-to-peer digital library federation.
All the tools are there. The same way that places like Stanford, Berkeley and MIT made MOOCs a thing, they can revolutionize scientific publishing.
There's already such venues such as JAIR for AI research (and that was set up looong ago without all the tech we have today), so it's certainly possible. It just needs to become the norm rather than the exception.
As a young tenured academic I have been systematically refusing to do reviews for non-open-access journals. (I'm certainly not senior enough to create a new journal and be taken seriously.) I hoped that this might give some ideas to my peers but it's seen at best as a weird quirk, at worst as a selfish move (because "the community" needs my help to review papers). Usually the answer is some variation of "yeah the system is not perfect".
The only argument I have seen which motivates researchers to care about open access is when it's mandated by funding agencies (e.g., in Europe, the ERC). Researchers really want to get these grants, so when these agencies talk about open access, they listen very carefully. It both makes me somewhat optimistic and quite cynical that the only way to move towards open-access seems to be via funding agencies (as opposed to researchers caring about the problem). Plus, for many agencies, "open access" here means "gold open access", i.e., continuing to work with the usual publishers, who make the articles available online, but move the "costs" (and huge margin) to the authors, with "article processing charges" of $1000-$2000.
I've been arguing that it's a coordination problem.
Aspiring scientists don't have the pull to change the system (if they move individually they only bear the individual disadvantages without bringing about the big collective benefit), and established scientists generally don't care or are too busy with other things [1].
That's why a concerted ("political") move like this cOAlition S thing is so important, and I find it very disappointing that academics speak out against it. I will read more to understand why.
[1] With notable exceptions, like Don Knuth getting the editorial board of the (Elsevier) Journal of Algorithms to resign and move to a new journal, taking the reputation with them.
I have to get as many publications into as highly reputed international journals as possible if I want to have any chance at getting a permanent position, and the vast majority of all reputable journals in my area are behind private paywalls. There is a maximum of 3 reputable, fully open journals and they are still not as prestigious as the established ones.
If I decided to only publish open access journals from now on, that would be immediate professional suicide. According to our institute's internal regulations, I'd be gone in two years from now, maybe even in one if they interpret their rules more strictly.
You're talking about eliminating, or at least sidelining traditional academic publishing. It's a completely different proposal.
there is no doubt about that. but maybe it's time to question how science is done in general.
in the end the goal is to advance our knowledge and bring humanity forward.
but instead of everyone cooperating to do just that, they are competing with each other, and try to outdo each other. a lot of energy is wasted in preventing others from stealing your research ideas and being the first to publish on a particular topic. instead of looking at the benefits of the research published in a paper, and whether the results can be reproduced, instead what matters more is how many citations the paper can get.
reputation has become more important than producing actual results. academics and academic institutions are measured not in the quality of their research, but in the amount of papers and citations they can produce, to the point that researchers who can't dedicate their life to their work, because they have family, or worse, are a single parent, can't get a job, let alone tenure, because they can't put in the time required even though they may well put in more effort than others into the time they do have.
so yes, i acknowledge that changing this is going to be extremely hard. but it looks to me like changing the way papers are published will be the easiest step, because the components that actually matter are distribution, which is technology, and reviewers, which are academics.
the only thing that i see publishers doing is to edit the journals and decide what to publish. but shouldn't exactly that, also be done by academics?
how about a model like stackoverflow? papers are published like questions, and reviews are the answers. readers upvote good papers and good reviews, so that the most upvoted and most reviewed papers float to the top. the citation count can be included in the score too.
greetings, eMBee.
The problem is that long-form works that require substantial analysis by experts (such as scientific papers read by competent scientists) don't work for a voting system like Stackoverflow. Internet voting works on things like comments of HN but not for 20000 word papers.
(I made previous comments about the limitations of voting systems to vet academic papers: https://news.ycombinator.com/item?id=15269673)
Basically, scientists are humans and human nature says they won't log into a system to upvote/downvote papers. Instead, they want a stronger signal than karma points before they spend (or potentially waste) their precious time on analyzing the merits of a long paper. That stronger signal is another respectable scientists that asks them to look at it.
scientists would focus on actual reviews and citations.
the only challenge i see is how we motivate people to write reviews. perhaps as some sort of a trade. for every paper uploaded you are expected to review 3 other papers. so that each paper can get at least 3 reviews.
there is also a potential for making a difference between public and private reviews.
private reviews would be like prepublishing reviews now, and public reviews like opinions about a paper after it's published.
The last thing we want is the general public voting on scientific research. Any results that don't fit their world view will be voted down.
I suspect such a system would devolve into clickbait and politics very quickly
There is another factor to consider - the outsiders. They need to know which research is good and which isn't and independent journals giving a stamp of approval to create "branded" science helps.
i would not trust a commercial entity whose goal is profit, to be able to decide for me what research is good, the same way that i don't trust movie distributors. i'll look for outside reviewers who earned my trust because they understand the subject matter.
For institutions, especially those in charge of grants, something more stringent and organized is needed than "I like that science guy". And those organizations will need to be payed somehow, preferably in such a way that it doesn't create too perverse incentives.
we pay you to do research, you send us your results, and we publish them.
or someone higher up, if the institutions get their money from the government, then that same government could fund a separate institution to review the results (so as to see whether the money is well spent)
there is still a room for commercial publishers, but instead of controlling access to the papers they publish, they get to republish what their audience thinks is interesting. so they are no longer gate-keepers, but add value instead.
and whoever thinks that added value is worth it, will pay for it. universities, and especially individual researchers need not pay for it, because they can go directly to the source.
magazines like nature or national geographic for example. but it could also be a company like microsoft that funds reviews of papers that are interesting for their business. or some independent thinktank. or the government of a different country.
everyone gets free access to the original research. that's what open access is all about. and anyone can fund reviews to suit their needs.
That's not true. The end goal of scientists is to get get grants to they can get paid. As they climb higher, they get more money for less work. Soon they get to attach their name ( sometimes in the first position ) to academic papers when they did no work on it what so ever. To do that they means they need to generate papers and publish papers.
Publishers are exploiting this.
> the only thing that i see publishers doing is to edit the journals and decide what to publish. but shouldn't exactly that, also be done by academics?
Academics do not have any interest in this. My wife was a PE at a major STEM publisher with a portfolio of dozens journals and supervisory responsibility for about a hundred. Academia's technological adaption ( which is necessary for efficient editing/producing workflow ) is terrifying. Corrections are done by hand. Proofs are faxed. Dropbox blows people mind. Papers that are accepted by EICs are unreadable -- EICs simply kick the paper to publisher to get it "produced".
If academics are not going to do publishers' job then publishers are going to control the process and decide what to charge.
no, there can only be one reason for publicly funded scientific research. to advance our society.
as for academics having no interest in technology. yes, that is a problem. but that is changing. the younger generations are more comfortable with technology, and i am guessing that if they don't use modern tools, it's because their supervisors can't deal with them, and they are not bold enough to change the status quo. but they will once they are in charge.
And that is not going to happen for next fifteen to twenty years.
Everyone realizes this. The problem is that you're abusing your position in this reputation system for profit. Do you think $40 is a fair price to read a paper from 1987? https://journals.sagepub.com/doi/pdf/10.1177/109434208700100...
It's not. Unless it's a way to drive people to buy subscription. That is: "pay $40 for a single article, or pay just $19.99 a month to access all of them". Which is also crappy.
> Obviously these opinions are in conflict with the business with which I'm involved, and are certainly not shared by most others within the industry.
This is true, unfortunately (same goes for media such as audio and video, sadly).
They absolutely don't. Every time this subject comes up, there are plenty of people wondering how expensive it can be to host a PDF. It's not; the price they're asking is not for the service of hosting a PDF, but for granting you the privilege of using their brand. A publisher has a monopoly on its brands, which means that there's almost no downwards price pressure.
Every time this subject comes up, there are plenty of
people wondering how expensive it can be to host a PDF.
1. why the system is as it is (and why it's hard to change)
2. what work are the publishers doing that is actually contributing value
The hosting cost is a point regarding 2, and making that point is consistent with understanding 1.
- why don’t publishers pay reviewers for the extensive time they spend reviewing the paper?
- why don’t publishers pay academics who edit journals, another time consuming task?
- why don’t publishers pay academics or institutions for a product they sell?
I can’t comment on your company, but the above is the status quo for most of academia.
Do you think this business model is ethical?
People trying to change or improve the publishing process _do_ understand academia - most of them work or have worked in academia.
The reason they're not engaging with you is likely that that your model is so obviously broken, and arguably unethical, that they don't want to help you or work with you - they want to put you out of business. Personally, I think that is an entirely reasonable response to the behaviour of academic publishers. Publishers have had more than enough time to reform.
As I get further into academia I am amazed at the amount of work we do that is uncompensated yet required of us by the antiquated system of prestige that you mentioned in your parent comment. Perhaps it isn't so much that academics are stubbornly committed to an outmoded system, it's that we literally have no time or energy to do anything beyond the bare minimum of meeting that system's constraints.
Given that the public for-profit publishers are reporting profit margins that put tech companies to shame, I don't really buy the argument that it's the academic community who are holding things back. The perverse incentive here is clearly concentrated on the publisher side, not ours.
I really appreciate that you are taking the time to respond to comments in this thread though, thank you for posting.
To provide a frustrating anecdote from the publisher's side: we'd love to heavily invest in launching new open access journals (which we do, but we'd love to do even more). The problem with launching a new journal (either subscription or OA) is that nobody will publish in it if it doesn't have an impact factor. Impact factor is controlled by a private, for-profit company (Clarivate) that's owned by a private equity firm. Getting an impact factor takes 3-5 years and also relies on the total crapshoot of what Clarivate decides to list or not list. So the prospect of launching a new OA journal is one where you are guaranteed to lose money for the first 3-5 years and then you have to put all your eggs in the impact factor basket, hope you get listed and receive an impact factor, and only after all that will academics choose your journal over any established legacy brand. And all this because at some point academia decided that they'd outsource academic career assessment to the magic number that is Impact Factor.
I also want to thank you and the other commenters for some good discourse here. This has been refreshing and I was only called an asshole once the whole time! But jokes aside, a sincere thanks :)
In the first year of my PhD the final-year student on "our" project, who I shared a bench with, told me all about impact factors and which journals he was hoping to get his paper published in.
1. Academia has a broken incentive model. I agree with you, I think this is a valid point.
2. Academic publishing is an exploitative monopolistic business.
I don't think the big publishers will be able to do much about (1), because they haven't done anything about (2) - which seems to me to be a much easier problem to solve, as the power is in your hands to immediately start paying reviewers for their time, as a simple example.
I do have an issue calling it a monopoly, however. At best you can call it an oligopoly. The top 5 publishers publish about half the total articles each year [1]. So half the research is published by a combination of hundreds of smaller publishers (both for-profit and not) or independent scholarly societies. And then within the top publishers, they are absolutely in competition with each other, which becomes readily apparent when you dig into the royalty deals that publishers offer scholarly societies for the rights to publish their journals, which continue to get richer for the societies (which poses a whole different interesting problem in terms of the collateral damage to modern-day scholarly societies if or when the business model blows up).
[1] https://journals.plos.org/plosone/article?id=10.1371/journal...
Regarding the monopoly comment: firstly, you have (usually) a complete monopoly on the content you provide. The same paper is available from one publisher.
If we talk in more general terms, then we're in the classic situation where the monopolist (or oligopolist, which is often used synonymously nowadays) pretends they don't have a monopoly, because they don't want to be punished or reformed. But, even if we took your viewpoint, that strength of market power - 5 companies controlling 50% of the market - is overwhelming.
In practice, Elsevier and Springer control nearly all Computer Science publishing (for example), so the situation is extremely bad.
It's clear that Sci-Hub is the only viable strategy.
why don’t publishers pay
It’s a society. You publish papers which get reviews, and you also work to review other people’s papers. Everyone contributes to make it work.
What about when the tax payers fund the research and then they have to pay to access it? Didn't they "already pay for it"?
But the question I was replying to was 'who is paying the reviewers'. The answer is that their employers are.
You 'pay' for reviews of your papers, by reviewing other people's papers.
All the publisher does is connect people up and produce the final product. You can argue that the cost to subscribe is therefore too high for that service. Fine, it may be so! But saying 'why don't they pay their reviewers' is to misunderstand what the entire setup is here.
It's pretty clear why this is a stupid system, isn't it?
Simply saying "but that's the way it is" is not an answer. It's the problem.
Maybe! I'm just explaining why the reviewers aren't doing the work unpaid on their own time, which is a misconception people seem to have. Fewer misconceptions is better for the discussion about the remaining issues like whether the system is stupid or not.
I don't recall seeing that misconception. I had a quick look in the comments here and I don't see it. Can you see any examples?
> No reviewers are ever paid either
> reviewers are never paid
> Nobody in the review or author role gets paid
It's all through this thread, and every thread on this topic.
The complaints are all about the publisher. Have you ever seen a complaint that's simply about having to do peer review itself without being paid?
> How do peer reviews happen, then, if nobody gets paid to do them?
They're not asking about the publisher paying. They're asking about being paid at all.
You claimed "It's all through this thread, and every thread on this topic", so it shouldn't be hard to find plenty of examples.
The example you gave is not an example of it. Notice the "if" in it? Go read that comment again, and read the comment it is replying to. The person is not expressing a positive belief. They're not familiar with the system, are trying to make sense of the parent comment's statement, and are asking how it works.
Edit: The bureaucracy of charging for reviews would be bad enough - but I suspect perverse incentives would soon arise as reviewing would soon be seen as a revenue generator to be maximised and a cost to be minimised.
This is inaccurate. This is not a part of a postdoc's job description, for example, in the UK. The university does not pay you to do this - you have to review on top of your day job.
Do you think academics and industrial researchers take holiday when they have to travel to a program committee meeting? No of course not, they do it during work time and are paid for it and their employer pays for flights etc.
While I was in academia I was specifically tasked with reviewing papers for an external conference, and so were all my colleagues.
Back to the topic: if he received papers to be reviewed, he gave them to his post-docs.
Second question is an interesting point: what happens if you don't review? I can tell you that it is certainly the case that a lot of academics don't do reviewing. Given the amount of reviewing I and my colleagues do, there must be others who are not doing their fair share (say 3 x your submission rate) of reviews. What are the consequences of not reviewing? Perhaps you can be seen as a freeloader, some social cost, but mostly people just won't know unless you're replying to them directly.
(Disclaimer: comfortably funded post doc here)
[1] https://meta.org/ [2] https://www.chanzuckerberg.com/science
That's not at all my impression of these discussions. If people thought it was just a technical problem, then they'd think it would be easy to change the current setup. If people thought it was just a technical problem they wouldn't have such a strong dislike for the publishers.
https://www.talyarkoni.org/blog/2016/12/12/why-i-still-wont-...
Springer, for example, has very relaxed and open copyright transfers. You basically retain the right to have copies on your website, on archives, etc., provided that they contain a link to the 'definitive version' of your paper. Elsevier does that to some extent as well, but Springer _also_ invests money to 'cross-fund' (if that is a word) books in fields that do not attract so many readers and that otherwise could not be published. So in that sense, Springer is giving back to the community and people are appreciating that more.
I agree, however, that there seems to be generic and unspecified hatred against Elsevier; having not reviewed for them or published with them, I do not have a properly-formed opinion here.
Personally I've lost count of the number of threads and posts where at least one person says "Publishers are in the reputation business."
The fact that you seem unaware of this is curious. If you genuinely think that plans to disrupt the industry are going to be based on distributing PDFs, it's possible you may not be as familiar with the discussions here as perhaps you could be.
Do you think it’s easier or harder today vs. 10 years ago for an outsider or upstart to create a cab or ride sharing company? How about starting a book selling company?
I’m not sure how those industries are better off except for the deeply-entrenched players that took over their respective markets.
Publications ( number, when, where, citations) is the primary currency/value in which one is judged within the peers in academia, and reputation outside the immediate academic community has a much lower weight. Whereas for online market places solid revune is the first priority and then comes reputation ( which is a means for the higher reveune). In academia it is the reverse, with reputation ( in a small clique) being the primary motivator, and funding being the means to gather it.
Personally, I do think the matter is complex. However, if you're trying to protect the current model by arguing that building a reputation costs money, you will not convince those who, quite reasonably, believe it should be earned by contributions to knowledge, instead.
The way I understand the debate about reforming academic publishing, it's about the morality of profiting from resources set aside for public benefit, rather than private (or corporate) profit.
Absolutely, huge coordination problem (prisoner's dilemma). Let's see what this cOAlition S can do about it.
Exactly. The name "publishers" makes people think that their primary business is publishing, but the main source of income is seeking rent on the use of their brand names.
A very specific example. In the US, there was a research team building a (post earthquake) hut that could be built with only plywood and also was built to be "tightened down" to resist earthquakes and hurricane force winds. It had anchors and was a really cool technology. I went to Haiti and the people there would rather live in tents temporarily and move into a concrete house because that's how you build a house. Compare this to Paul Farmer, who was on the ground and built a network of medical facilities by including the people he wanted to help. He's been (by my standards) wildly successful.
It's easy to see a change that seems obvious from the outside, but if you don't get in and work on the ground floor to disrupt with people and change culture you're making your job way harder. Get in and talk to people, figure out why, and remember that people are just that people. Politics / Culture are unavoidable. Sometimes the easy part is the technology and that a'int easy.
But what about the humanities and social sciences, which are typically not funded by government or foundation grants? We currently have a system in which the expectation of the academics is that they can publish for free because the universities pay for that cost via subscriptions. Changing to an author-pays model, which Plan S seems to push the industry toward, doesn't work for a lot of academic fields. There are certainly alternatives, like university libraries converting some of the funding they currently use for subscriptions to cover publication costs, or entire governments covering all publication costs for every academic within their borders. But it's not as easy to see exactly how the non-zero cost of publishing is covered outside well-funded disciplines.
Yes, thank you for restating the obvious. Research should be open, that is what we want. Off to do useful things now.
Prove it. Get out ahead of these issues. Add value. Progress marches on. Lead the way.
Let's assume (for scope of this comment) the replication crisis is the biggest threat to everyone's reputation.
What would a solution look like?
More access & transparency & accountability.
Imagine a scholarly clearing house that facilitated the existing processes and workflows.
Something like github.com for warehousing all the data. Something like a collaborative editor. Something like scholar.google.com to better find & forage for stuff. Something like linkedin.com for researchers to connect and share their thoughts.
Maybe even a brokerage for grants, connecting funders with applicants, help administrate the administrivia.
Publishers would still publish journals. The cream rises to the top. aka curation. As Scott Galloway likes to say "Information wants to be expensive."
TL;DR: Innovate to add value in a changing world, instead using the status quo to continue rent seeking.
--
"It's like discussing politics online."
For policy work (vs electoral politics), I very much agree. Policy work has it's own replication crisis.
(Lowly soon-to-be-postdoc here; a professor would be able to say more.)
That's almost true. Elsevier once surprised me with a $100 Amazon gift card, which they said was in recognition for a particularly good job of peer review that year.
I'd laugh at the prospect of Elsevier rewarding me with "recognition" -- certainly something I would not put on my CV -- although I did take their money.
Another time, the American Mathematical Society (which is non-profit, and perfectly ethical) asked me to do a book review, and then gave me $150 in bookstore credit afterwards -- again by surprise.
My understanding is that book authors can make a reasonable amount in royalties (a few thousand, anyway). For journal papers, as you say I've never heard of any author getting a royalty.
This is not how I think about my job. Rather, the university pays me a salary, and expects me to engage in "scholarship" and "professional service" -- and I get to decide where I think I can make the biggest contributions.
This luxury is not available to everyone. Universities are hiring more and more non-tenure-track instructors, at lower rates of pay, with less job security, with higher teaching loads, and (usually) without any research expectations. In the future, we might be obliged to think about our jobs in more transactional terms.
But for now, I have the luxury of not having to think very much about questions like yours.
Probably the parent has experience in CS or similar where reviewers are never paid.
We get this idea from basic fairness within the rules of the larger system. Clearly, there are private companies that make money from publication of those papers. It follows that in a fair arrangement of some sort, they would have acquired the necessary rights from the original authors for some compensation or profit sharing agreement. That they don't actually do that is indication that arrangement is not fair.
[1] https://www.economist.com/china/2018/11/17/tsinghua-universi...
When I start thinking about what we lay public have been denied, in welfare, progress and life, just because scientists don't share the knowledge the lay public pays them to produce, the word hilarious isn't what's coming to mind.
Ironic for a scientist to rail on anecdotally and off topic about a problem they play the much larger role in deciding the fate of.
Knowledge of science among the general public let alone of how this sausage factory works is however unfortunate, not the key to fixing the problem.
There are a handful of different key roles and interests competing to influence the outcome. Researchers, universities, publishers, for profit businesses, etc.
It’s not that complicated. There’s the logistics, economics, and best guess modeling of any proposed changes.
Then separate from these technicalities people everyday are making value judgements and prioritizing principles against personal interests.
I’m not trying to advocate my opinion here in a comment of what these stakeholders should do. Only that these individual decisions, judgements, and will to act on them play a far greater role than public knowledge of who gets paid to review a paper.
I wouldn't see any incentive for any work to be peer-reviewed, ever, then.
We tend to think of journals as being run by their editorial boards -- i.e., by scientists in our field. The publishers are in the background, not taking a very active role, simply skimming off truckloads of money in the process. In particular, peer-review requests tend to come from scientists whom we know at least by reputation, and often personally.
When I spend serious time peer-reviewing a paper, it will be on a quality paper on a subject that I enjoy reading about anyway. (If I get asked to peer-review a paper that looks uninteresting to me, or something outside my interests and expertise, I'll either decline the request or write a quick report recommending that the paper not be accepted.)
It's interesting work, it's important for the health of the field, it helps me support other researchers in my field (if I like their paper and give it a positive review), and it helps my own reputation (the journal editors are often bigshots). From this point of view, it's a win-win-win-win.
Finally, paying for peer review doesn't make much sense because the time required is highly variable. I might take between 20 minutes and 30+ hours to review a paper, and I wouldn't necessarily expect the editor to be able to predict how much (even within an order of magnitude) in advance.
However, I see no reason why high-profit publishers should be able to insert themselves as middlemen and engage in rent-seeking behavior. I hope we figure out a way to give them the boot, and then continue approximately as we have been doing.
That seems kind of harsh?
I'll then read the introduction. If I don't have sufficient expertise to referee the paper, I'll decline the request. If I do, then I'll see what the authors have accomplished. If, in my judgment, this doesn't rise to the level that the editors asked for, then it doesn't take me a long time to decide this and say so.
Conversely, if the introduction does impress me, then I will want to check the proofs in very close detail. In this case I will commit to writing a detailed report in the future.
It's quite common for papers to be declined from individual journals; it's happened to me plenty. There are tons of other journals out there; you can submit somewhere else. And when I decide that a negative report is called for, I write it right away, so as to not keep the authors waiting forever.
You can always put work up on arXiv if the goal is priority or distribution. Peer review is, in principle, designed to ensure that work is new and true (and that what is new is true). It’s not meant to fix flawed work, but to make good work better.
...I fear however that my hopes are in vain
I've had many paper rejections, none because of false or uninteresting claims but almost all because "it's not good enough for this prestige journal" or "it's too specialized for this journal". I expect a certain number of those, because I submit to reach journals now and then and hope I get a reviewer and editor that like the work :) If I wanted certain publication immediately I'd submit to lower-tier journals first, which I have also done when I just needed something out.
Economics and linguistics also publish their own journals which costs some money but significantly less than Elsevier's subscriptions (I believe linguistics' is 300 USD per year). There's probably enough momentum behind middlemen-run journals that prevents this from happening more en masse.
Paying per paper maybe not, so maybe paying per hour makes sense?
To respond succinctly to your comments: money isn't the only incentive in life, so it seems weird to not see "any incentive" as soon as money is taken out of the picture.
From the outside, it is easy to forgot the extreme extent to which academics are motivated by (or view themselves as motivated by) principles of "good citizenship". This is selection -- the people who aren't this way, often do not get hired or promoted. Also, many academics (at least in pure math) view motivation for money as a "lower order term". It's a good thing for them, because for much of your long career in academia there is little you can do to impact your salary, besides applying for a job somewhere else. For me it's always been: each year you get some maximum possible merit raise of between 0% and 4%, depending on external economics that the department has no control over. Academic book royalties might also raise your yearly salary by 2%. Being highly money-motivated in some parts of academia would end up being very frustrating indeed. E.g., even when I've got a big NFS grant, that doesn't change my salary one bit; instead, it changes how many students and/or postdocs I could support.
There’s also an ulterior motive for some lines of work, where sending a paper to be critically reviewed by rivals is a mechanism to torture test the work & conclusions.
This can be taken too far (especially in biology, the glam fluffing and $million additional irrelevant experiments are legendary), but in principle, if even your most motivated critics can’t find a fatal flaw in your work, it’s a reasonable bet that the work is sound. At least, that’s the principle. Editorial overrides sometimes break this safeguard, though (lord knows I’ve seen a few).
Well, yeah, but I think what he's saying is it's probably the case that a large number if not majority of academic make the decision to go through the grueling PhD process and give up years of earnings in their prime for things other than monetary gain. But of course people can become disillusioned later down the road.
Too many examples of “oh well, let’s try the next journal” and not enough of “gee maybe we should fix these glaring flaws”. It’s not because I’m a nice person; it’s because I would prefer the literature not to be a toxic waste dump. Nobody is entitled to be published anywhere. In an ideal world, doubly blinded review would become a part of the published record. At some journals it already does.
The reviews offer valuable context, which is often sorely lacking in high profile venues. (The canonical examples of STAP and arseniclife come to mind, but also much more subtle details where an overall sound paper somehow only gets cited for the one shaky assertion in the results)
JMHO
Especially for a group of people where many of them have taken a pass on more monetarily lucrative careers.
Maybe it gives you the opportunity to network with higher-profile contributors to your field, if you offer to peer review their work (though to be honest, I don't know how the peer review process works, and if you can even "offer" to perform it for a specific paper).
Peer review is not the same thing as replication, which is replicating the results of a paper after it's been published (e.g., confirming some groundbreaking finding). Peer review happens at the stage before publication of the original paper. Researcher(s) submit the paper to the journal. The journal editor sends the paper out to some reviewers, who review the paper (this is the "peer review" stage). Pending reviewer feedback and editor approval, the paper is published.
Edit: also the "benefit" that rsa4046 refers to probably doesn't mean networking. AFAIK, reviewers are always anonymous to the authors (which can generate its own problems e.g., if the reviewer gets a paper authored by someone he/she doesn't get along with). The benefit being referred to, I believe, is that of learning to write better reviews, and having reviewed other's work, learning how to improve your own.
The authors most likely won’t (can’t) ever know that I agreed to review their paper; it’s more of a good citizen affair. Sometimes, afterwards, it will become apparent that a particular referee was someone familiar. I would like to think that the original poster was imagining something along those lines, but your take is probably closer to the truth. Oh well.
The ultimate point being that the group that pays for the labor does not get the benefit of the labor: each member of the public must individually pay for access to the article. This situation is absurd: they have already paid for the article's production.
This is why the most powerful open access initiatives are being driven by grant agencies. They are in a position to unequivocally state that the research they are paying for must be open to the public since they grant the money on behalf of that very same public.
Lowly soon-to-be-postdoc here
Congratulations :)
As one of those academics...I keep getting requests to peer review, I respectfully make clear I don't review for non open source journals anymore. Same with publishing. I'm not tenure-track so am not primarily evaluated based on output.
Publishing is broken, but it is really just part of the broader and even more broken nature of academic research.
Contrast this to the peer review culture of popular open source projects - major pull requests have extensive and transparent dialogue, and disagreements known. Meanwhile there is no barrier to releasing anything new.
I am also a developer. There are far more people that can give feedback on most code, and also it doesn't take a lot of apparatus or money to get good at coding. Also CS/OSS is relatively young when compared to other disciplines. We've almost always done things with honest and sometimes brutal feedback. Even academic research is usually announced/shared at conferences. Look at what happened when Nature tried to make an AI journal.... [1] I think there is still value in someone getting paid to manage research and referees and ensure a high quality product. Open peer review is just going to be a lot harder for some of these disciplines with limited experts.
[1] https://www.sciencemag.org/news/2018/05/why-are-ai-researche...
- Many research fields have anywhere between 3 and 10 groups working in them.
- A review always transpires the background of the reviewer. You just cannot "mask" the shape of your knowledge around a highly-specialized subject. This includes your approach to the problem, the issues you are most interested in (and hence know more about), the references you give, etc.
With a closed system, you only get to see:
- Reviews of your own paper, without knowing who wrote them.
- Reviews of the papers you review (there are usually 3-4 reviewers per paper, and you get to see the other reviewer's reviews and who they are).
With these pieces of information, academia is already full of grudges and strong-arming around. Here's an anecdote:
I was once in a conference, and met a colleague that made the effort to approach me and comment that he was great friends with my advisor (who didn't attend), and just wouldn't stop praising him. After a while, he switched to ranting about the review process, and how a specific reviewer was a moron that wouldn't understand anything and so on. Of course, that reviewer was my advisor and I did know but he obviously didn't.
Given the high egos involved in academia, I am pretty sure that if there was a track record of all reviews, researchers would figure out who reviewed their papers. The backlash would then be ugly, and the entire ecosystem would end up more corrupt that it already is (imho).
I've been on all sides of the fence, and I just don't see a better solution than just having secret reviews. Nice words don't matter (much, to most people) when the e-mail starts reading "We are sorry to inform you...".
If all reviewer comments were collected and then published at the same time to prevent them influencing each other, how could that make the situation with respect to “frank and honest” reviews any worse than it is already?
Are there entirely secret journals where nothing is accessible to non-contributors, for the same reasons? I suspect the answer is no, which is why these feuds often play out in attack-counterattack sequences of published papers.
Yes, and this already happens (hence the grudges I was commenting about).
> How does maintaining secrecy for the per-paper improve the situation with respect to “frank and honest” reviews?
By increasing the uncertainty. If I am 95% confident it is you who screwed me last year and 50% certain this paper I got now is yours, I will look at it with a harsher attitude than if I'm only 20% and 20% certain.
If you were to publish all reviews then researchers would have much more information to convince themselves that it indeed is "that guy".
It is sad, and I personally despise it, but that's what it is from my personal experience.
PS: I quit academia after getting the PhD, and this was among the reasons for me to quit. The other major reasons being that I don't want to keep relocating somewhere else in the world every 2 years until I'm in my 40s-50s, I don't like the overselling and result dishonesty in general, and I actually enjoy working in industry too).
Ouch, that’s a good (and depressing) point.
I think it is unlikely you can separate the two precisely. If it be so, mathematicians can replace doctors who make diagnoses. I don't think all empirical facts can fit nicely into a formal proof system.
But still, any effort to reduce the burden on the human referees would be welcome. This said proof system for peer review would be most useful to math and theoretical CS (though not so much to biomedical sciences).
We could at least have the data and the statistical analysis (code) accompany the paper.
Dialogue is an unmitigated good, but dialogue also tends to be dominated by the normative voice. Where science is different than OSPs is that OSPs need to work with what exists now, whereas science needs to engage in tension and informed dialogue in a more foudnational way..
And keeping everything together in one (de)central database would make it possible for people, so inclined, to annotate other people's work with new references to support or debunk the work long after it was published, or to clarify ambiguous language, etc. Those annotations, too, could be subject to filtering as needed.
People could build reputations and whole careers around tying up loose ends instead of the "publish or perish" grind.
How do you propose that is done?
For my job I build neural networks for text processing. I spend a lot of time reading papers in the field.
And yet if I look at something in an adjacent field (even something as close as something like open information extraction) I have trouble telling which papers are important.
How on earth am I supposed to tell if something in a further removed field which attracts more crackpots (say probability theory or something) is a fringe theory or a breakthrough from a new author?
I'd note the example of the Gaussian correlation inequality[1] where even people in the field weren't aware it had been proven for 3 years after publication[2].
[1] https://www.quantamagazine.org/statistician-proves-gaussian-...
[2] https://en.wikipedia.org/wiki/Gaussian_correlation_inequalit...
What I'm imagining is a sort of layered approach with raw "article" (or some other unit) data at the bottom and indexing, tagging, clustering, filtering, reviewing, commenting, linking, etc. layered on top with possibly many implementations to choose from.
The analysis/filtering would also apply to people augmenting the data, so if some users are really good at tagging certain types of junk as junk, you could easily filter out that junk. If there emerges a cluster of users who keep tagging certain interesting material as "woo" then you could filter them out or even use them to discover interesting material.
I doubt there's a silver bullet (at least today) that could reliably distinguish between unconventional-bad and unconventional-good work, but keeping the baby and the bathwater together opens the door to such an algorithm in the future.
I can see two noncynical reasons for it one. One is educational purposes. Two would be quality control - if it is already debunkable or widely known there is no point in publishing in Nature "thinking done in the brain".
The cynical abuses and perverse incentives however are myraid and academic politicking is already infamous. I don't know what solution will work but understanding the system goals and functions should help decide what to replace and what to keep.
Because I am not at all prepared to assess the quality or accuracy of a quantum physics paper and I doubt a molecular biologist is prepared to assess the quality or accuracy of a sociology paper. In what I have seen there are a lot of cargo cultists who think kthey understand a field, and use big words, and just don't get it. I feel that there should be some filtering function to remove that.
As an example:
This guy: https://www.mountainproject.com/forum/topic/113602967/near-m...
This guy: https://www.reddit.com/r/badphilosophy/comments/7x3t1g/stand...
This guy: http://ecclesiastes911.net/
and
This guy: https://arxiv.org/search/math?searchtype=author&query=Simkin...
are all the same person.
What would help? My opinions...
-FOSS publishing -Abandonment of journal metrics, they really serve no purpose besides trophy hunting at this point
-Stronger transparency initiatives
-Public review (c.f. Lim)
-better journal metadata (it affects citations)
-Greater shaming of misbehavior of ALL types
-Elimination of stupid policies like 'issue lengths' The article either should be or should not be published...saying there isn't space in a given issue is insane given that >>90% of article access is online.
-a hell of a lot of older faculty retiring out of the way of science.
I'd prefer to keep article lengths in place. It's easy to write a lot of text (as quite a lot of high school students understand), but it's challenging--and very important--to be able to convey information in a very concise format. The process of trimming down the text to squeeze it under the page limit is very useful in getting the ideas contained in it to a more refined and easily-understood format.
I agree, in theory, with your point. I think the unfortunate thing that is lost in most efforts at concision are lack of detail rather than fluff. Fluff is bad, but I think detail is important...especially detail on how things are situated in prior literature. As a reviewer, I have often found there to be a lot of articles using 'strategic concision' to gloss over not doing things properly, or flat our not knowing what you are doing. Saying 'we used method x' means I have to trust you did it properly...or infer from other things how well you did it. Supplementary material could be a potential route here, but I just sent an article back for the second time. The first time they did the 'we did x' and I asked for more detail. The second time they said 'we did x by doing y and z'...but y and z were very very wrong.
There are places where being able to explain concepts concisely are important or a viable tactic. Journal manuscripts are (in theory) a permanent archive of new knowledge for human society and have never struck me as the place were concision is a relevant parameter.
The inclarity, and multiple verticals seems more like an example of Goodhart's law than anything else.
Summarized, the goal of academia is and (for most of time in whatever form it took) is to produce and disseminate new knowledge for human society. Creating new knowledge is what we now call research. Publishing papers isn't a raison d'etre...it's a metric for that.
I'll note that Ernest Boyer is far more articulate on this than I am...
There are a lot of things that only exist as suspicions and intuitions inside of a researcher's head, and that sort of information is much more likely to come out in a back and forth reviewing a paper than in actual published literature.
It's based on open standards and an open platform.
W3C Web Annotations: http://w3.org/annotation
About Hypothesis: https://web.hypothes.is/about/
“A platform for scholarly publishing and peer review that empowers researchers with the
Autonomy to pursue their passions,
Authority to develop and disseminate their work, and
Access to engage with the international community of scholars.”
Harry Crane, an Associate Professor of Statistics at Rutgers, is one of the founders and a good follow on Twitter.
Its how science worked for most of its history. Really, Elsiever is responsible for the profusion of people publishing fluff to weak journals, subsidized by the government.
PeerJ and PLOS One are good starts, but until academia is no longer a slave to "impact factor," it will have limited effect.
Replicating researches? That's how we prove findings are not wrong or biased, usually.
The top 50 research universities could get together and decide to no longer give consideration to pay-walled articles. It wouldn't affect them AT ALL...the research they do is already treated as pretty much cannon no matter where it appears. Everyone else would follow, and the journals would quickly respond with change.
Yeah I know it's unlikely but they have enough social capital to actually have an impact.
The simple fact is that like any replacement system, there has to be a reason for it to exist. And for a social network, there have to be network effects in place. Fragmentation is a problem. "Oops, we ran out of VC money..." is a problem. Hell, "We got bought by Spinger" is a problem.
In the very long term, they don't have to exist along side each other. In the medium term though, they do. Otherwise, it's like going from "I'm going to quit my job tomorrow with no savings!" to "I live sustainably on my off the grid farm" with no plan for how to eat in the middle.
I think we could start fixing things by incentivizing good peer review. There’s a lot of ways to do this and I’d be interested in a discussion of different schemes. It will by no means be easy but I think that’s what needs to happen.
It has been amazing to come to understand just how effective the corporate takeover of universities has been in the thinking of senior faculty. I have had faculty complain about policies that they literally are the only one's empowered to change because it has never occurred to them that they actually have power.
Conferences, generally, are a better path to science because of dialogue. Having a conference with 2500 presentations because everyone needs another CV line item don't accomplish that because no one is in the room.
(The good stuff starts where Robert Maxwell appears)
When a paper is seemingly not available online, I've always gotten a free copy via an email to the author... And then there's sci-hub. It's not the way it should be (i.e. you shouldn't have to hunt around for publicly-funded research), but at least it's something.
40% are very normal and consider healthy margins for many businesses. The business with largest possible profit margins I know of is political donations. For mere $100K, one can own vast public land for mining and selling resources worth billions for many generations. That's 1000000% margin for you.
> the business of publishing tax-funded research and then selling it to tax-funded institutions has produced the most profitable industry in the world
How is this not illegal?
That statement is misleading. Pblishers sell participation badges to the article authors, the authors put these badges in their CV, and when the time comes to ask for money from the governmnet, the gal with the most badges on her chest wins it.
In addition, the authors have the right to upload a preprint (i.e. a copy identical in content) to a preprint server - virtually all journals allow it nowadays - although it doesn't happen.
So, there is in theory no reason why anyone should go to these websites to download a copy. In practice, it is convenient for the parties involved (researchers & funding agencies).
I don't see why that should be a requirement. So long as papers are made available for free, why should it matter if they are also available in a paid journal?
Alexandra Elbakyan has done and will do more for science than this rent-seeking asshole will ever do.
If there was a way to open source curation and not lose quality, I can't imagine anyone would disagree with that - even the people who work at these organisations.
Unfortunately, what you end up with is arxiv, while very useful for sure, has no curation.
Everyone just keeps bleeting - we want free! But they don't bother to think about how to do actually do it. How to ensure quality curation remains which is absolutely so critical to the advancement of science.
I've been fantasizing about free everything since forever. Who hasn't? But at some point we have to stop trying to fantasize our way to results.
This is like underpants gnomes logic - steal underpants .. .. .. .. quality curation!
Also, MrGunn is a moron commenting on that thread. Why not just hand out free subscriptions to rare disease patients? What a trivially cost less PR move.
Science without curation will fall to corporate shills and propagandists with political axes to grind. It's already bad enough but with no gatekeepers at all it would be a total free for all. Tobacco would become good for you again, homeopathy would work, etc.
Curation can be democratic, but if so there must be a well thought out procedure for electing curators and a constitution or set of bylaws that is hard to amend. Science has a lot of power, so any scientific curation system is going to come under constant attack. We are living in the age of information warfare.
Your argument equates to "they have maintained their grip, therefore they're providing real value".
Their stranglehold seems to come primarily from academic incentives to publish in prestigious journals.
> The idea that after 23 years of internet they've maintained their position via smart deals is ludicrous.
Perhaps someone has claimed that, but I don't recall ever seeing that argument.
Grant agencies receive money from taxes and those funds are devoted to science instead of other public projects. That the output of this work is not generally available to the public is the problem.
Most researchers will never feel this problem since their libraries will have access to the journals that they need. Publishers in general will just charge for access and not see the issue. It is only when you go out as a member of the general public and attempt to read some journal article, presumably funded by your taxes, that you will realize that you haven't had a voice in all of this.
Shiran Pasternak What is a reasonable profit margin for a platform for communicating publicly
funded research? Non-rhetorical question.
mrgunn For a publicly traded company, the answer is "what the market will bear".
Krista Jamieson offers a great perspective.
The same goes for everyone who works there. They have either found a way to justify that what they do is ethical and okay, or would quit. That's just human nature.
Either way, I don't think he's helping the image issue.
Sci-Hub hosts 85% of the articles published in paywalled scholarly journals on a shoe-string.
I'm not saying current publishers are great at these things, just that they should not be forgotten and should be improved on if a new order was to take shape.
I remember but can't find a citation for an apocryphal story that the administrative overhead of metering, tracking and billing phone calls in the Ma Bell days was more than 50% of the cost.
We publish here, and the stuff that is good rises to the top with upvotes from our peers, and it is all subject to review in the comments where many an interesting discussion is had.
We all gain by sharing ideas as widely as possible. We just need a way for that to happen.
An academic article is not a blog post. Also, only the most popular posts get discussed here -- tons of stuff is lost if it's not initially popular. Timing your submission allows for gaming the system.
All scholarly articles need to go through peer review and receive honest, constructive feedback -- especially the bad ones, and Hacker News (and Reddit and Slashdot and any system like that which doesn't assign reviewers to articles) isn't very good at that.
Which is also a system that can be gamed, by the way. For example, by the reviewers:
- "Oh dammit, we got scooped... better use the insights from here to help us past our hurdles, while stating this work is not original enough to be published so we can quickly submit our own and get published earlier after all!"
- "I see that this paper belongs to Camp A with regards to these two competing hypotheses explaining unexplained phenomenon X, I am in Camp B. Therefore, I will consider this paper rubbish by definition."
- "I am an old researcher who does not believe in some of the newer findings that contradict longstanding assumptions that I "grew up with", and therefore will never allow this paper to pass."
- "I am a very insulated researcher who cannot make sense of this multidisciplinary work, therefore reject the parts that lie outside of my field as rubbish."
- "This paper is a by a researcher competing for the same position/grant/etc. as me. Better make sure they don't get too much credit."
- "I'm not sexist/racist/etc but... totally clearly biased in my judgement of this paper because..."
Now, I'm obviously describing worst-case scenarios here. But my point is: it's human politics all the way down.
My argument is that a blog aggregator style system is a bad fit which doesn't meet the requirements we desire from a replacement of the current commercial publishing scheme.
If you tried to use them for things that people care more intensely about, like determining what gets to be part of The Academic Literature, then they would instantly become the subject of concerted efforts at manipulation. Even now, HN has to deal with voting rings and the like; it would be much, much worse if people's careers were riding on it.
It's Goodhart's law: as soon as a measure becomes a _target_ it rapidly stops being a good measure.
This will come as a surprise to both my PhD advisor and my current advisee
My dissertation was based on a highly speculative new treatment, in a field where mathematical models are not widely used. One of the potential answers was "Yes, this should entirely change the course of care for a particular disease."
My current advisee's first paper is, essentially, "Has the field been doing this wrong this whole time?"
Note that I publish regularly in the good society level journals in my field, and on occasion in the splashier journals, got a tenure track job at an R1 university, and have enough grant funding to cover my lab.
Also, your assertion that "incremental" research can't change the research status quo is flawed.
It's so obvious you're being ripped off. What the heck scientists?
No wonder people believe in conspiracy theories, with a bunch of scientists denying them the knowledge while a bunch of business suits hoard the cash.
If you know of one for CS, do comment.
ArXiv is not peer reviewed.
First step is to put the pdf into the zip and have existing tools be able to navigate the hierarchy. Could include notebooks, bibtex, tex, data, images, etc.
And semantic scholar is in exactly the right place to institute this evolution.
Take a P2P program similar to Kazaa or BitTorrent to share paper pdfs and datasets.
Next to these files also share indexes, derived data for searching.
Next to that, also share user generated review comments and endorsements to implement peer review and quality filtering.
Finally, digitally sign all these data to guarantee provenance and credibility of the data found on the network.
Let me know what you think of it. I’ve been loosely working on a prototype application that does all that for a while.
Academia is largely a social construct, that's why it's hard to replace with 100% technology.
Since any derived data would be signed and linked to the identity of its author a default filter removes anything that is not transitively included in your academic network.
Reviewed articles would just be “review count > N and reviewers in fieldNet”
An online taxpayer-funded archive could be created/designated by statute. It would require that such articles be published there (before and after reviews). Publishers would pay to have access; their prices would reflect that cost and a cost-plus profit. Privately-funded org's could decide which game they prefer.
Make a fine memorial for Aaron Swartz.
That's an extraordinary claim to make. I would seriously doubt the "best" profit margin cross-industries is capped at 40%.
Interesting. Does anyone know what the volume is like?
Assuming there is no money driven agenda, even the reputable journals have the issue where much of the content doesn't get peer reviewed. The findings are far from concrete.
But money driven agendas definitely exist too, and without basic vetting a journal finds itself hosting bs lobbyist material with sloppy "science" finding favorable results for anti-GMO, alternative medicines, organic farming, anti-vaccine, etc. The problem also exists that these entities are effectively funding their own journals to buy "legitimacy"- and they certainly would pose a threat to any journal that doesn't have high vetting standards.
As of yet, free access journals aren't reputable or actually desirable because they don't have adequate quality control. Free access to information is a worthy and noble goal, but we can't render the information useless in the process.
JMLR http://www.jmlr.org/ is quite successful. There are some fields, such as machine learning, that are not dominated by for-profit journals. Why is this possible in some fields and not others? My answer would be that it is possible in all fields, but incumbency advantages can be very strong and coordination across academic volunteers can be more difficult when they are individually less secure.
The 'organic' products may be better on balance, but we don't actually know that, because the body of knowledge isn't nearly as developed yet.
That's my layperson's understanding, anyway. I could be wrong.
Using only "natural" pesticides because they are "healthier". This is unsubstantiated, again it's just an appeal to nature. It can become problematic because natural pesticides often aren't as effective, thus certain farms/crops require using an excessive amount- way more than they would if they used the more effective synthetic pesticides. The synthetic pesticides are of course accused of being unhealthy (because unnatural)- though this is generally untrue. Furthermore, even the natural pesticides cause pollution problems, and overusing them isn't good.
Because the organic lobby succeeds by the appeal to nature, they are also anti-gmo. Which is a big problem.
While there are entirely organic farms, a lot of organic products are really not even what they advertise themselves to be, as many organic farmers don't actually run organic farms: they run normal farms with a section of it that adheres to organic standards. These standards are pretty arbitrary, except for the fact that organic farming is a significantly less efficient use of farmland, which is going to become a problem within the next century. We need to be making efficient use of our arable land, as we don't actually have much left on the planet unless we start turning national parks + protected areas into farmland.
Largely, organic farming is a faux luxury built on anti-scientific ideas that does nothing except waste valuable land resources, sold to relatively privileged people who are willing to pay a premium to feel better about what they eat.