Darpa Wants to Build a BS Detector for Science | Hacker News Reader

Darpa Wants to Build a BS Detector for Science

wired.com

72 pointsmcone20 comments

In the fourth paragraph, Dr. Adam Russell (the interviewee) is quoted as saying: "I wouldn’t characterize it [as a bullshit detector], and I think it’s important not to". Then the author titles his article "DARPA WANTS TO BUILD A BS DETECTOR FOR SCIENCE".

That's not just weasel-y journalism, it's downright disrespectful to the person you're interviewing. If I was in Russell's shoes, I'd be insulted.

In my experience it's completely par for the course in science journalism. At least his contrary quote was included!

Assuming that he actually said that quote or something similar to it. It's really hard for me to trust quotations in articles after what I've seen.

The intelligence community works fairly hard on this, with only moderate success. But they have active opponents trying to fool them.

One of the CIA's checks is to ask of information that confirms other information, "is this from a different source, or is this from the same source via another path"? That's a problem that could be addressed on the web, and is helpful at detecting "fake news" and hoaxes. A graph of which stories are derived from what other stories, and when, gives a good sense of where something came from.

Google is terrible at finding provenance. They're likely to list the popular clickbait site, even if it's a scraper site, rather than the actual source. Sites which are listed as "news" get special treatment, but Google doesn't find the original news story, rather than its indications. Google crawls the web so frequently that it could get a handle on provenance from timestamps alone.

Clickbait title, but the issue of reproducibility on social science studies is well worth investigating.

What the article wants is something that I'd say couldn't and shouldn't be done (in terrible writing: "some sort of machine with, like, a slot for feeding in journal articles. And two lights on the front: red and green. Ping or bzzzt.)

What the investigators say they're working on goes from good suggestions (pre-registering a research plan (to ward off accusations of P hacking) and making full sets of data and the code used to analyze it available) to bad ones (Social networks) - if you're an outsider like Perelman you're out.

Building a bullshit detector is easy. Building one with a low false positive rate is hard. A low false positive rate is also extremely important because in many fields truly novel research findings are often dismissed and it takes a generation for them to become accepted. See for example quasicrystals. Given how hard making new discoveries is, and the fact that the peer review process is already stacked against new discoveries (as opposed to follow ons), I don't think this is a good idea for many fields. On the other hand, if you know an experimental methodology is flawed, then you can pretty much throw out all that research since the scientist didn't do the right experiment or they didn't do it with enough power (social sciences I"m looking at you).

Why not use crowdsourcing, and add some inputs to the detector? That might make it more reliable. The crowdsourcing could be limited to peers, which could increase the reliability further.

There have been some efforts. For example, there was a betting market on the psychology reproducibility project, see http://www.pnas.org/content/112/50/15343.full

Gedankenexperiment: construct such a machine and then create an adversarial network that attempts to fool it.

What do we do with the examples that come from the adversary?

Do the best we can to dismiss them as irrelevant while simultaneously using them to argue that the project needs additional funding.

As in "The project is fine! We just need to work out this thing".

Slight variation: how about an adversarial network attempting to fool scientific publishers directly?

I reviewed a paper for a little local conference last week that I think was probably machine generated, or the person who wrote it was on some very heavy medication. There was a similar one last year which I also rejected. I suspect that the team is trying to get them accepted so as to debunk the conference, which is small, but I like to think reputable - perhaps this is a widespread effort though?

Read "Truth Machine" by Halperin. Might have broader than intended consequences.

Oddly enough, here's a situation where headline's idea sounds better than what's discussed in the article.

The idea of a "confidence detector" seems both Orwellian and AI-complete.

But a "bullshit recognizer" seems simpler and practical. A device the recognizes contradictions in text, conclusions unrelated to evidence or similar "egregious" things could be useful.

It might even be a trainable thing. You have N papers with M passage-combinations that show the claims of said paper are bullshit and develop a finder/recognizer. The problem could wind-up not that different than sending Watson at Jeopardy.

The drawback, of course, is the thing would have to be taken no more seriously than a grammar checker - it just serves to point reviewers at potential problems rather than serving as proof of a problem.

Wouldn't, for example, set theory, be registered by such a machine as bs, owing to Russell's Paradox?

Well, if the mechanism was smart enough to flag naive Cantor/Frege set theory as contradictory by formulating Russell's Paradox, the thing would be pretty smart indeed. Naturally, you know Russell's Paradox only applies in that naive case rather than Zermelo-Fraenkel or Von Neumann–Bernays–Gödel theory.

However, I'd imagine the mechanism as having more modest goals - flagging the kind of language that's used to shore up papers with little logical basis, notice when hypotheses aren't used for conclusion or when conclusions are hypotheses, that sort of thing.

This is interesting. I'd think about submitting a proposal for my own projects in progress if it actually made sense to do so. Anyone with experience in that area - is it worth the time for a single dev to submit their proposals?

actually, after reading the PDF, it sounds like they're just asking for ideas, so it probably doesn't make sense to submit anything that actually exists (unless I'm reading it incorrectly)

It's not entirely out of the question.

Some of the other *ARPAs have "proposer days" where you can go and ask questions.

thanks, I am looking into these. here's one link for anyone else interested https://www.iarpa.gov/index.php/working-with-iarpa/proposers...

specifically, this one looks interesting, with a deadline in the next couple of weeks https://www.iarpa.gov/index.php/research-programs/amon-hen/a...

I would highly recommend not spending time as a single developer submitting proposals, and instead start trying to build relationships with some of the winners/participants of past IARPA tournaments. They regularly bid on these IARPA programs and have some level of success. Getting on one of their teams as a SME or with a specialized offering is a great first step. If you're in the area, the proposer's days can give you an opportunity to get up in front of everyone and talk for a couple minutes about you and your capabilities to get your name out there.

Reply on news.ycombinator.com