Hacker News Reader

This may be intentional - for most academic publishing venues, their target audience is other researchers of that domain. It's counterproductive to optimize an article for a lay audience if everybody in the actual audience has (or is working on) a PhD in that domain and is generally expected to be (or become) up to speed with the terminology and earlier results on that topic. The journal or conference is generally not meant to communicate their results to wider audience, it's generally a tool that some particular research community made for themselves to help their research by an exchange of ideas and results. An article in academic publishing is meant for someone who will use that to further their own research in a related field - and the needs of a person like that are very far from a lay audience, they will want to see entirely different things in that paper - you want the paper to focus on the novelty, on the "delta" between this result and what was bleeding edge a few months ago; not spend two thirds of the paper describing what was already known before.

There are publishing venues intended for a lay audience, but most academic publishing is not, they have incompatible goals.

Unfortunately, the 'lay' audience is often not all that 'lay.' For example, you might be working on a phd in an adjacent field to a given paper and still have a hard time coping with the sub-field-specific jargon. This happens all the time in math research, and contributes heavily to silo'ing of subfields. This silo'ing is actually super counter productive, and can lead to the same things being independently rediscovered, and important, useful discoveries simply failing to be communicated across silo boundaries.

I'll echo this for biology too. Often, the jargon is the exact same word, but used in a very different way. A quick example is the word 'vector'. In math, a line in space, in genetics, a vector is a virus. There are many other examples, especially with proteins, where those acronyms are reused many times, sometimes inside of the same article.

I was talking to a student the other day. He and I took many of the same classes in different semester in which we were assigned to read articles and then present them as part of class discussion. Typically this ends up being a talk all about the figures of the paper. He mentioned that most of the time, with bio papers, you have to read the article at least 3 times before you can start to grasp what is going on with it, not to mention the online methods section that goes on for ~20 printed pages. Often, the figures are quite clever and really do a good job at conveying the data in a great way, but they are presented as 'in-field' and the graphs, the diagrams, the data, etc. will be very very difficult to understand from a 'novice' point of view. For example, they will present the data out in a polar plot where the 360 graph is known in the field to represent only the arc of sensation out of the right 7th whisker of male mice under anesthesia. In that (contrived) example, you really have to dig about in literature to find this out and if you don't know that then you'll end up confused and really frustrated. Essentially, if you are a person that is somewhat interested in a very closely field of research, you have to spend hours upon hours trying to parse out some piddly little paper. It really is that bad. I don't know what that is, but it sure ain't 'science'. A paper in a closely related field should take you no more time to read and understand to the point of being able to tell someone else than the time it takes to eat a sandwich, a bag of chips, and drink a coffee in total.

Every obstacle is an opportunity, in this case a service to unpack papers for someone not in the field.

Oh man, check out youtube or every other podcaster for that. Like, millions of videos of people 'talking' about papers, though they run into copyright stuff all the time too.

One thing I've learned is that someone in that position should be looking at PhD theses before journal papers. They can be long enough that they function much better as introductions.

It's a sad industry that has enshrined culture that the more experienced/senior a person is, the lower quality their work is, because they have to make it artificially inaccessible to make themselves appear more elite.

This and review articles are the only two avenues currently available. I'm glad to see the original post here making another effort to provide resources.

I agree with your basic premise: Researchers are not supposed to be writing for a lay audience. No arguments here.

But too often, the language (both english & mathematical) of research papers is not intended to enhance academic communication. Rather, it's a faux optimization to game peer review or make results look more "technical" in a way that is actually detrimental to communication.

In effect, the language is counterproductive. And the time spent on faux conciseness is wasted; much like that old quote, "I would've written a shorter letter, if only I'd had more time."

Out of curiosity, which field are you working in precisely?

In my experience (in type theory / programming languages), the main problem is that a lot of papers are just badly written. Good papers really do use precise language to be easier to digest for other people in the same field.

The reason why researchers write for a niche audience is because they are maximizing publication count. We're essentially targeting the papers for their reviewers. If researchers did not care about publication count, they would target the widest possible audience so that their results would have the most impact.

It's rather the other way around, it's that reseachers read only/mostly works designed for a niche audience; and thus only value such works. We measure "the most impact" as "the most impact on us", i.e. how much the work impacts the research community.

Works have an impact if I read them - a publication that targets a wide audience and thus by necessity is three times as long and "too introductory", is a publication that won't be read (by other researchers), won't be cited (by other researchers), and thus won't have an impact. If lots of laypeople read and understand it, well, that's nice, that's how textbooks work, but it doesn't help advance the science. A citation in mass media doesn't count as a citation. If someone silently adapts your method in industry, that's nice, but it doesn't really matter as well - that's a sign that your tech is mature, but "impact" measures the impact that your paper has on future progress - if the next thing doesn't rely on your paper, it doesn't have an impact on your research field anymore. Reviewers are actually representative of the intended audience of readers, so targeting for people like them makes sense.

"Maximimizing publication count" seems to be the accepted ground truth, but I'm still somewhat surprised researchers don't write for a larger audience because one would think that expanding your audience beyond your sub-field will have a higher chance of garnering citations.

Maybe the "lottery" citation factor isn't worth the extra effort required to normalize in-field shibboleths. Then again, if maximizing citations truly is the overriding publication incentive, I wonder why low h-index researchers don't pursue this route and 'upgrade' their existing papers for a wider audience wherever possible.

And perversely, this makes it so that people feel compelled to add mathy things that add no additional explanatory power to the results.

I've frequently heard complaints that people in some CS subfields in particular will just add complicated math because they have to rather than because it helps. It sucks because the authors don't have that great a grasp on the math anyway, the readers don't care about the math -- they just care about the intuition and process, and communication suffers overall.

My best papers had at most like two equations, and they were just there to clarify models for fits and the like in case the reader wasn't familiar with terms like "Arrhenius relation". So, you know, to actually make reading it less burdensome rather than just making it look more sophisticated.

Exactly! In the end, we ended up just publishing on arXiv.org and writing up a blog post [1]. It's not earth shattering... just an entirely new idea being demonstrated for the first time. The mathy background certainly exists, and was explained in other cited papers, but wasn't necessary to explain the new ideas.

[1] http://www.hizook.com/blog/2015/08/10/mobile-robots-and-long...

I've definitely had reviewers tell me in a paper that I needed to make my math more formal and really prove things, when all I really wanted to say was, "We take the limit of foo as bar goes to zero, and get baz. Look, baz is interesting!"

Here are my thoughts based on my experience with peer reviewed publication.

There are two high level criteria for publication: novelty and difficulty (this is in my field of Programming Languages and Systems so keep that in mind).

The novelty requirement is important and I trust that you satisfied it but (as you pointed out in a child comment) you may not have met the difficulty requirement and the reviewer did their "best" to articulate that in a way that isn't the all together ridiculous "not hard enough".

Naturally we might wonder why "difficulty" is a requirement at all. Shouldn't the importance or impact of the work be the thing that matters regardless of how difficult it was to achieve? The problem is that it's _extremely_ hard to know what work will be impactful and so reviewers, who have to reject something like 90% of submissions, use the heuristic of "difficulty" to estimate.

This is a problem to be sure but I think it would be a problem in other settings as well.

I wanted to echo this experience. I agree that reviewers are generally looking for novelty and difficulty. Unfortunately, difficulty can translate into a system where reviewers create small fiefdoms: This isn't difficult enough for my journal, so I'm going to reject it. I say fiefdoms because I don't often see criteria for what difficult means, so it's left to individual reviewers and the editor to decide. Anyway, this creates some perverse incentives where it's now necessary to make a piece of work appear more difficult than it really is.

That said, I don't really think this is all that unique to academic publishing. At some point, I think we've all met someone professionally who makes problems sound more difficult than they really are and seen this person rewarded by management because they're the person who can work on really difficult problems.

I'm more on the theory end of PL, but in my experience program committees generally regard difficulty as a negative for a paper. The main thing PCs want is for papers which make progress on important problems. Now, you can do this either by (1) attacking existing open problems, or (2) by finding new important problems.

It's easy to tell that a long-standing open problem is important, but generally these problems are open because they are hard, and consequently the technical sophistication for new attacks on them is generally very high.[] New problems can often be solved with relatively simple techniques, but it's also difficult to tell if a new problem is genuinely important or just some random thing made up for submission.

With a low acceptance rate, PCs tend to favor type (1) papers over type (2) papers, because few people feel comfortable arguing for a paper that might* be doing something important over one that is doing something important. This is widely seen as a failure mode of low acceptance-rate venues.

So over the last several years many of the top PL conferences (such as POPL, LICS and PLDI) have moved to a multiple track format in order to increase the number of papers they accept. This is precisely to give program committees the freedom to take chances on "risky" papers that might become important, or might become footnotes.

[*] Occasionally, someone will solve a difficult open problem with simple techniques, and these papers are immediately accepted with much rejoicing. It's easy to have your cake and eat it, when you have two cakes...

> in my experience program committees generally regard difficulty as a negative for a paper.

I don't know if we're talking about the same type of difficulty. I don't think program committees see the difficulty of the problem being solved at the core of the paper as a "negative" necessarily?

> The main thing PCs want is for papers which make progress on important problems. Now, you can do this either by (1) attacking existing open problems, or (2) by finding new important problems.

My earlier note aside, I think this is an extremely important point. It seems to support the idea that "impact" is hard to gauge but much easier when the community has already arrived at some form of consensus on the importance of a problem.

I don't know about the details of the OP's paper, but I'm not sure "difficulty" is the right word... Mathematical rigor is needed in many domains. In my field, "not enough math" could well signify that the description of the experimental results is sloppy, or that you can't check whether the results presented are sound.

> A lot of this seems to be a problem with academic publishing, peer review, and the pedantry contained therein.

Ayup. It's too annoying to translate the academic speak, but you can shortcut it.

For example, anything CS nowadays (I include crypto in this) must be 1) publicly available on the web, 2) come with running code that runs some basic tests, and 3) be limited to a single compilable file. Failure in any of the criteria means I move on. If the source code involves anything like "configure" it's an immediate fail. If the code passes all three of those criteria, it's probably more useful to read the code rather than the paper.

I used to love ISSCC for the reason that they used to demand both A) a die photograph of a chip and B) actual oscilloscope traces. You can't hide when you have to make test equipment produce data. Sadly, they got rid of that requirement in the late 90's, and the information content of the conference suffered correspondingly.

You must have a very narrow definition of CS, because several branches of it have nothing to do with code.

>> Failure in any of the criteria means I move on. If the source code involves anything like "configure" it's an immediate fail

What do you mean by this? Surely some things require some configuration. Don't they?

Not as often as you think. And there are multiple levels to this.

1) CS papers are normally about a single or logically-related set of concepts. If you can't reduce the code to a single file, the paper is probably too broad or you are trying to hide something.

2) If the author wants me to take time, it has to be really easy to try. "Configuration" is asking me to make choices that I really have no clue about a priori. Now, maybe there is a "heavy duty" version as well. But the "straightforward" version needs to be really straightforward.

And you would be surprised how often you don't need the heavy duty version. A nice example of this in crypto is DJB's "TweetNaCl". It gets you 99% of what you need until you understand what it doesn't get you. And, it's so damn easy to include that you get very little resistance--try using tweetnacl vs libnacl vs libsodium in a project. You'll pull your hair out if there isn't a magic package already for libsodium while tweetnacl can be dumped into embedded microcontrollers without pain.

3) If the author wants longevity, simplicity is his friend. Can you even begin to run a "configure" script from even 5 years ago let alone 10? Yet, I can download an ancient Fortran 77 code file and compile it today pretty easily.

Configuration is not your friend.

If what you write is difficult to understand people will assume you're smart. If you've ever learned a foreign language you'll recognize this: Blithering idiots talking in that language sound smart because you're having a hard time understanding them.

Academese exists because it works.

In my experience, papers often take more than just two submissions to be published. It's often a question of finding a good community fit. One paper I coauthored took 8 years and probably 10 submissions before it was finally accepted.

Length restrictions can be a bummer, but if you have a publishable result that you simply can't squeeze into 10-20 pages (depending on the venue), typically you split it in to two or more publications. This has the added advantage of ensuring that each published unit is a smaller, tighter piece of work. I don't think it's just pedantry.

Many people would say that if you submitted 10 times, you are playing the system. You've had 10 sets of reviewers spend time to review your work, even though 9 of them told you that it's not worth publishing.

I guess that's the advantage that tenured researchers have over everyone else--you can't run out of career before you published all of your papers.

I don't really agree with the "playing the system" sentiment. It isn't as if we took the same paper 10 times and just submitted it until it stuck somewhere. Each time we got valuable, actionable feedback, which we then acted upon; the final paper was almost unrecognizable from the first submission except in the core ideas. We gained a lot of experience as a team from this process, and the end result is a much, much better unit of work entered into the literature.

None of my other publications have taken 10 submissions, so this case is also probably an outlier.

Depends on what reviewer actually told you. There is not worth publishing and there is not worth publishing in this journal. Those two are not the same.

Moreover, there is level of subjectivity in peer review.

Is there anything like an academic paper marketplace, matchmaker, brokerage?

Not to my knowledge. Gaining familiarity with the different venues in your subfield is an important part of the graduate (and beyond) education.

Carl Sagan was famously denied tenure at Harvard and membership in the National Academy of Sciences for his science advocacy. Different takes on it range from "jealousy" of his peers over his broader popularity to anger from them for his making science accessible and explaining concepts in a way that allowed readers to understand them:

https://en.wikipedia.org/wiki/Carl_Sagan

I used to see this with my peers in computer programming in the 1990s. There was a lot of anger and jealousy when everyday normal people started putting up websites. Several of my CS friends were of the opinion that this was almost polluting the WWW with bad code.

> quoting a reviewer: "This paper lacks math."

Why didn't you place the additional detail in an appendix, so as to not detract from the main points?

Most journals have very strict page limits.

(Also: The mathematical underpinnings are related to radar systems, whereas the application is in robotics. The amount of background information to bring a roboticist up to speed would've consumed entire papers or books -- some of which were previously published by myself & other coauthors.)

In my field (physics) I find that an increasing number of journals allow supplemental materials. You provide the main story in the text, but can add a lot in the supplemental materials. For example, suppose we have discovered a new material with some exciting properties. There is a fair amount of characterization that we do just to convince ourselves that we made what we think we made. A lot of this can go into the supplemental materials for those that want to make sure that we have done the standard sanity checks. A long time ago, you would write a short letter, then a longer report. That fell away, but I am really a fan of supplemental materials when used correctly.

"I know of at least one "seminal" robotics paper that was rejected from top venues multiple times for "simplicity" (lack of mathematics content) that went on to become a foundational paper in the field after appearing in a lower-tier venue years later."

Curious. Which paper is this?

I'm guessing the original SIFT paper?

I'd be interested to read that paper, if you're willing to share it.

I think it's this: https://arxiv.org/abs/1507.02373

Direct pdf link: https://arxiv.org/pdf/1507.02373.pdf

Correct me if I'm wrong, but if you don't value what a publisher is offering, you could publish your paper on your own, make it accessible to an audience of your choosing with a delivery of your choice (for example, the wonderful work of Aphyr, here: http://jepsen.io/).

I don't get why people would try to publish in high-tier venue. To me it seems much more about polishing one's ego instead of improving the research quality.

As a young researcher, you need the publications when you apply for academic positions. High-tier venue publications are easily recognized as such. To recognize high-quality work published in a low-tier venue, the only metric is citations. Accumulating citations takes a long time (you might not have) and is helped by active marketing (you might not do).

This is more about quick-polishing your image than your ego. ;)

Researcher has to have some amount of authority in his field to receive grants. Publishing in high-tier venue is a good way to get that authority because it is how people-with-money judge researchers :-)

Well I really enjoy reading articles from top-tier venues (PL theory). It means that committee does a good work filtering and improving articles, definitely not without mistakes.