Comment by lucb1e | Hacker News Reader

Comment by lucb1e | Hacker News Reader

I do not recognize the problem the author talks about, but it seems weird. From the article:

> Prof. Aaronson, given your expertise, we’d be incredibly grateful for your feedback on a paper / report / grant proposal about quantum computing. To access the document in question, ...

It seems odd to want feedback and then ask someone to go and register somewhere, probably requiring to accept a bunch of legalese in the privacy policy and terms of service... Just attach the document you want feedback on, right?

At least if I'd email someone (out of the blue or an acquaintance) for feedback due to his expertise, I'd be grateful for the time taken and try to make it as easy as possible to do.

Edit: it has been made clear to me that it's not about individuals contacting the author, it's some big corporation that probably sends this out, probably in an automated manner. I still don't understand why anyone would bother with this when "peer reviews" can happen between "peers" (i.e. sending each other documents for review, rather than going through the middleman that everyone seems to hate such as Elsevier, if blog posts linked on HN are to be believed).

This is the norm in academic publishing. Not only do publishers (often for-profit publishers with huge profit margins) ask us academics to critically review submitted manuscripts gratis with short deadlines, but they also want us to use their poorly designed, byzantine online systems to do it.

It's most fun when you are just trying to decline this invitation because you do not have time and are asked to create an account and password (with stupid password rules, naturally), and fill out an amount of personal information just to do that.

You make it sound as if you are required by law to do this. Short deadlines, for free, on poorly designed systems, behind a login wall. Then you just don't, right?

And if it is a government agency asking you to review a grant proposal? Besides the obvious service aspect, you don't want to unnecessarily offend someone with money, that you might have to apply for yourself someday.

I understand you're also helping colleagues, but in the case of wanting peers to review your paper it's a simple matter to send emails with PDFs to those peers.

But I'm not sure how this world works anyway. I read about research in books that refer to it, Hacker News and Reddit that links to it, etc. but never in a magazine. Yet the magazines seems to be what everyone is aiming for to be published in, they are apparently some big deal. I suppose there is some reason why "peer reviews" are not between "peers".

The peer review is supposed to be anonymous. So what happens in practice is you upload your paper to a central website from which a set of reviewers can download a copy for review and give feedback with comments, anonymously. This is also why there is a login system: to ensure the comments are from actual peers which have their own papers up in the repository too.

I'm not saying this is the best way to do it, just how it is now.

Also, if you want someone to just read your paper, sure, email is fine. One does not exclude the other. The central system is there to ensure that should any doubt arise over the quality of the work, the feedback can be read and interpreted.

Again, there might be many other more high-tech solutions out there to solve this problem.

Ah, anonymity is the issue here. That explains it a bit. Thanks for replying.

The process, as I've experienced it, is:

1. Authors write paper.

2. Authors send paper to journal or conference address. (...using a web form)

3. The editor gives the paper a once-over and decides to send it out for review.

4. (Editor sends links to reviewers, who log in, read paper, and leave comments.)

5. Lather, rinse, repeat until accept/reject decision is made.

They used to send pdfs, but now use web systems for roughly the same reason HN isn't a mailing list.

Note: the last time anyone let me review a paper was a long time ago, when they sent paper forms to fill out. Parts in (...) are how I assume it works now.

That's exactly what the original post said: he won't do it any more.

It's not about the individual, it's a lot of people I hear about this. Yet it is apparently the common practice. I'm asking why.

I guess it is because people see it as necessary in order to build relationships, and have a name. But it looks like it's time to draw a line.

For the same reason it happens in other websites/services: to collect personal data and to subscribe the user to their spam^H^H^H^H mailing list.

Eventually they will bar you from publishing with them. How bad that is depends on the journal and what you intend to do with your life.

Speaking as an outsider who mostly knows how academic publishing really works from reading occasional blogs that complain about the subject, the entire system seems like something straight out of a Douglas Adams novel.

"Why don't you just stop doing X thing you don't like?"

"Because if we do, they won't let us do Y thing we do like. Or, well, we would like it, except the way they make us do it is so awful."

"So stop doing that, too, or go do it your own way instead."

"But if we don't do Y thing we don't like, then they also won't let us do Z thing. Incidentally, we don't like that one, either. But we need it to be able to do X."

It is totally bananas.

There are two questions: why review at all, and why are the review systems so crappy? For the first, the whole system in the academic world is based on peer review, and I think peer review has mostly proved its worth. For the second: 90% of everything is shit, and the academic world is struggling with the legacy of for-profit journals and trying to get better systems in place. But opting out of everything crappy leaves you with almost nowhere else to participate.

If you are early in your career and want to make your name heard, then you do accede to all these requests (which probably is why they get made with ridiculous requirements in the first place)!

I think people are too polite sometimes. These requests are an abstracted "GFY" so I think the exact proper response is a non-abstracted "GFY" - with a CC BY-ND 4.0. Basically it's a reply that's congruent with the request, but without the bullshit.

I do see why they ask for that (although it sounds like their systems could use some improvement).

Imagine you trying to get someone to review a 1,000 line patch. If you can just get them to log in to GitHub, they can use all of the pull request commenting tools, and it'll be easy. Otherwise, you have to mail them a big .patch file, and then somehow manually manage comments across all of that. The assistive tools are more convenient.

So then give them a one-time login ticket, and let them upgrade to an account?

Not saying you could do that with GitHub, but there's design space for it.

And add that you have dozens of such patches that you need people to review, and their reviews have to be distributed to a committee of senior developers once they're received, and so on and so on...

A journal may have dozens of submissions at various stages of the peer review process. Each requires two or three peer reviewers, who are busy academics who will often fail to meet the deadline. The editor has to track all of these, follow up with lagging reviewers, and then put together all the reviews for a paper to make the final decision.

Doing all that by email would be a mess. Reviews would get lost, the editor would lose track of which reviewers were sent which papers, and so on.

So the online systems make sense. But they all suck. I've reviewed for several journals which use the ScholarOne system, for example, and despite all being hosted centrally, they inexplicably require separate logins. The website is a clunky piece of crap. If they just had an automated system email me a PDF and ask me to email back a review, and had their script grab reviews from the inbox and format them for the editor to review, it'd be a lot easier. Or if they emailed me a personal link that let me view the paper and leave a review, no login required. But alas, they must overcomplicate it...

Linux kernel patch review is over email like this. It seems to work. I subscribe to a few lists and it makes it easy for anyone to chip in.

It's a bit burdensome on the patch submitter, to collect the feedback into one place, but not that much. At least the burden is on people who want to get the patch in.

Github accepts Google account sign ins. I personally no longer register for services that do not accept my google account as a form of authentication. I wish not to leave even more online accounts and passwords scattered over the web. With google auth, they do not get a users passwords. Without google auth chances are most people will fall for password-reuse.

Last time I saw a Google account log-in, the remote service wanted full access to my contact list. That is, it was demanded that I betray third-party confidences to friends in order to gain an authentication to the service.

Fuck no.

Here: https://ello.co/dredmorbius/post/vv0bq6oia_06z_yjnmjwzw

I wish we still had Persona.

But there aren't twenty GitHubs. There are three or four, and their are hosted versions of those that are slightly different.

If publishers all used the same software package for content management, we might be somewhere. But every company with an IT department seems to build their own Content Management System at some point. I joke with coworkers that we would solve the developer shortage in north america if we figured out how to stop everyone from writing their own CMS.

When asked to peer review something for academic publication, the PDF of the paper is, in my experience, never ever emailed.

Do you know a reason for this? That seems it would be the lowest-effort option for all involved.

Note that the person asking for the review is not the author of the paper, but the editor of the journal or conference. They have several hundred submissions to keep track of at a time. They need to assign two or three reviewers to each paper, and then find alternative reviewers when those decline. They need to collate all the reviews for a paper, use them to decide whether to accept/reject/accept-with-modifications the paper, forward the reviews to the author (except the parts that the reviewer has marked confidential), having stripped the reviewer's identity. If the authors upload revisions, all the reviewers (but not anyone who declined to review in the first round) should receive the revisions, and so on.

I assume they are trying to keep the documents in question confidential. Email is not confidential.

EDIT: Folks email is not secure. Even if someone doesn't have access to login to your mail account. Emails in transit are insecure.

Using outside systems to send confidential data is common practice and has been for years.

In practice, it's no more confidential to use a system where you have a separate, high-security system, that sends out new passwords via email on demand.

I disagree. A sysadmin is not allowed to take a peek in emails unless there is a reason. Or if I create a folder named "private" on my work computer or work email, they aren't allowed to look in there without a good reason. But I suppose Dutch privacy laws aren't universal, so I don't know how that would be in your (or the author's) country.

And if you want to take matters into your own hands, you use PGP. But outside of the computer security business I guess that's mostly unused.

Not allowed, or not able to?

If what is not allowed is not enforced by effective technological measures, or at least logged and reviewed by other trusted parties, then it may as well be allowed.

At my company we trust our people and have a web of review. However, we try never to have anyone be the final trusted party - even the boss.

Email in transit is not secure.

This is not strictly true with US businesses and academia. My last place of employment was a university, and per our legal, all data going through the University email or computers is University property; while there is a pledge from IT that all email is private and will not be accessed except for specific circumstances, these circumstances could be pretty much anything and had no relationship to any existing case law outside of what legal felt was necessary to use as reason for why we could access it.

But the pledge was just a formality - should there be any interest to check the mail, there was nothing technically stopping anyone with the required access except for someone else with equal access having a problem with it. Likewise, various management offices had no issue with submitting email fetch requests for the simplest of things, with date ranges exceeding two or three years sometimes for what turned out to be incredibly minor reasons.

There really is no expectation of privacy with academia in the US when it comes to University owned property or services.

(The only reason I found out about any of this from legal was because I asked for clarification on our "duty to report", and one of the lawyers was almost excited as he told me about how we own everything legally.)

Almost all online login systems offer password reset by email. The moment another human has free access to my emails he effectively owns nearly all my accounts.

You're technically correct (which is the best kind, of course), but for peer-review of scientific papers it's not really relevant.

There's nothing _private_ about the paper, since the author intends to publish it, or about the reviews, since they're going to the author anyway. Only the identity of the reviewers is confidential, and for that email is perfectly sufficient. Well, a standard email client would be far too error-prone; you'd want something like a blind mailing list just so the editor doesn't click the wrong button and accidentally unmask the reviewer.

If you're using large email providers, every step of the email transaction--SMTP to the outgoing server, SMTP between MX's, IMAP/POP to the mailbox server--is conducted using SSL (assuming the end user hasn't elected to avoid SSL when configuring email clients, which is actually hard these days). Yes, SMTP between MX's are routinely sent using SSL: Google reports that about 80-90% of such traffic is using SSL.

OpenPGP is probably more secure than whatever system they are using.

>It seems odd to want feedback and then ask someone to go and register somewhere, probably requiring to accept a bunch of legalese in the privacy policy and terms of service... Just attach the document you want feedback on, right?

It's odd, but it still happens all the time.

Reply on news.ycombinator.com