Hacker News Reader

Comment by j-pb | Hacker News Reader

Your argument doesn't hold economically nor based on the state of the status quo.

Economically it is infeasible to politically play, social engineer, or bribe thousands of reviewers in a publicly inspectable web of trust. It's easily done so and done in a secretive environment with a handfull of people that all know each other, i.e. the current state of publishing.

The current paper review is a farce, you just go conference and review shopping till it gets accepted, papers get shitgunned into half a dozen journals with minor changes, and reviewers often have no clue about contents and just check the general form, wave papers from friendlies through, or block competitors based on personal vendettas and funding rivalries.

Publishing and review in a decentralized anonymised web of trust secured by crypto is probably the only valid use case for crypto and blockchain foo that I know of.

View on HN

> Economically it is infeasible to politically play, social engineer, or bribe thousands of reviewers in a publicly inspectable web of trust.

You're replacing peer review with a social media following. A large social media following is absolutely something you can buy.

Which, by the way, has already happened. The easiest way to get an R1 job is to build a large twitter following. Getting a top 5 CS position without at least a few thousand twitter followers is basically impossible these days. High-prestige academia is just a high-brow social media influencer these days.

None of this is meant as a defense of the old system, btw.

That's why it's important to make it a web of trust and not a plain "upvote"-"downvote" system.

E.g. publishing a paper that gets positive reviews gives you some form of trust token that you can spend on other papers in a positive review.

Ideally there is no single global metric for trust, but trust is computed relatively based on who you trust, so that everybody gets a personalized quality and relevance score for each paper.

In a way you could also personalise your definition of trust.

E.g. do you want to rank papers highly that have positive reviews by a handfull of highly trustwothy individuals, or do you trust papers that have many positive reviews.

That's just another popularity contest with all of the exact same perverse incentives. How do you think people get on conference committees / journal boards?

We should stop worrying about popularity contests (citation counts, prestige-signalling conference committees/paper venues, etc.).

When we decide whether to retain an academic as a visiting scientist, consultant, etc., I am always the voice in the room encouraging folks to disregard everything except for one question: after having read 3 or 4 of this person's most relevant papers, do we think they can help us with the specific problems we face?

When I sit on grant review committees I ignore all the academic prestige shit and ask: is this a real 10-15-30 year problem for a field, does that field matter to the funding agency's objectives, and does the description of the plan of attack have a good chance of success? I don't need to evaluate popularity contests to answer those questions.

Popularity contests are low-effort short-cuts for the low-brow scientist/program manager. Best to ignore them if you have real problems and need real solutions.

The point of the subjective and individualized scores is to avoid that, but fair enough, if we could get academia to switch to an open funding scheme without the current paper mill incentives more power to that.

I fear that the scientific community is not mature enough to do this properly and truthfully without some gamification though sadly, and funding commitees will want some kind of metric for "non-technical" people.

But maybe the only solution is to pass out funding based on random lottery tickets.

> paper mill incentives

In the USA, publishing a large number of low-quality papers is at best unhelpful for getting grants. It's probably (very!) harmful.

> funding commitees will want some kind of metric for "non-technical" people.

The stakeholders within funding agencies are always technical. The external stakeholders -- congressional staff and corporate executives -- could not care less about citations/paper counts.

Based on your view of how these things work, I'm a bit curious about your background here. Have you sat on a funding committee for a public funding agency? Do you know the names of any program directors? Have you served on an academic grant review committee for your employer? If so, I'm really curious to know what field you're in, since things appear to work quite differently there.

> Economically it is infeasible to politically play, social engineer, or bribe thousands

If this were true, we would not have snake oil salesmen, pyramid scheme leaders, drug lords or youtube celebrities

You just want to turn academics into yet another category of celebrities that must game social media. We have enough of that shit

We have that in academia already. It's games day in and day out.

We need those games to be transparent at least.

And a web of trust is not the same as social media.

Its not what we have in academia.

And please show me where do we have a functioning web of trust

> Its not what we have in academia.

If you are blind to this then you are either in a very sheltered environment or you're part of the problem.

We haven't build such a system yet, which is exactly the critique OP had.

The broken funding process based on citation count and journal impact deincentivizes building and participating in such a system.

It's all self enforcing with the publishers reaping in billions.

It took years till european academia finally started to rebell against elsevier, and even now open access gets sabotaged by golden path open access publishing.

If web of trust was a workable system, i wouod have been used in some industry by now - engineering, law, anything.

But you criticism is based on a fantastical benchmark that was never seen in practice

https://en.m.wikipedia.org/wiki/Web_of_trust

OpenPGP for example.

Oh, you think its functuonal?

Send an email signed with openPGP to your bank, can they validate your signature?

Have you ever tried to get an OpenPGP key to sign your email, do you know what a nightmare it is? Because i have, and I couldn't, I have also built kubernetes clusters from scratch, and it's easier.

Okay, say you got a key, send it to your IT department, what % of IT proffeshionals can use OpenPGP, 2%? What % of 'normal people' can use it, 0%?

That is your great success, 0-2%.

That's way worse than the system academia has now, it might be crap, but its not 98% crap

You asked for a successfull web of trust, not a great example of an encrypted email client.

How is it succesfull? What did it succeed at?

I just outlined how no-one is using it and no-one is capable of using it.

What is the metric and benchmark of success you are using where OpenPGP with it's 0% adoption is successfull, but the way academia works currently is not successfull?

OpenPGP's web of trust model is broken at the fundamental level.

> Economically it is infeasible to politically play, social engineer, or bribe thousands of reviewers in a publicly inspectable web of trust

Elon Musk hyperloop whitepaper pipedream...he made it in one night, probably on heavy drugs. It got press for 8 years, still getting press.

A bastardised version of Hyperloop got hundreds of millions in public funding in Las Vegas and Los Angeles for what essentially amounts to a large underground car garage with skeets. The company selling this bastardised version just raised 700M in equity.

"infeasible".

When you open the floodgates of the court of public opinion...it becomes the game of the person who screams the loudest and is able to con people with the most confidence or has the loudest megaphone.

History has shown us time and time again that the best governance system is a bunch of elder statesmen who are terrified of being judged harshly by their peers long after they are gone.

For example people still say about Newton that his track record wasn't immaculate because he wasted time looking for hidden messages in the Bible or trying to turn bronze into gold.

Same for Einstein, his track record isn't immaculate because he was wrong on quantum-mechanics.

Elder statesmen who dream of having the track record of a Pythagoras, that's the perfect governance system, that just requires finding them, doesn't require crypto-tokens or blockchain or any other flavor du jour tech.

"History has shown us time and time again that the best governance system is a bunch of elder statesmen who are terrified of being judged harshly by their peers long after they are gone."

Those elders have a hard time steering the ship in the storm of technological change. Faced with that, at best, they will most likely try avoid implementing anything radical (due to their bias for what's familiar to them) and will instead attempt to integrate the new with minimal change to existing system (which makes sense as a strategy, if that happens to work). At worst, they will ignore or actively resist change, betting on the likelihood that it's about some overhype at play and things needs to settle a bit before being taken seriously. As sound as this may be as a governing strategy (in general, not only for handling things in academia), it comes however with the inherent risk of inflicting debilitating effect to governed realm's progress. To avoid this, the governance has to include visionaries, and a governing body including only (although assumed visionary, yet just) elders, with a blindspot in judging much of what's new and unfamiliar, will not do.

A public web of trust network based on asymetric crypto with weighted reputation scores between scientific entities, is an entirely different thing than trending on twitter.

Besides, the "paper" you cite is not a paper at all, yet plenty of universities had their fun with the concept, so if anything it shows that it's possible to get people excited to do study reproducing work after all.

Reply on news.ycombinator.com