Hacker News Reader

Comment by lol768 | Hacker News Reader

It's unfortunate that the app was licensed out to a commercial vendor ("Peak Brain Training") when the study received funding by the National Institute for Health Research (via the Department of Health and Social Care) which is taxpayer funded.

Why wasn't the code open sourced and made available under a free license?

View on HN

I'm disappointed by this too. The FTC fined Peak Brain Training competitor Lumosity $2 million because it "deceived customers about the cognitive and health benefits of its apps and online products."[1] I was more hopeful about the app if it was standalone.

As for the code being open sourced, the results of the research should be publicly available but the material of the research is an asset of the university (like how the computers and beakers used in other experiments wouldn't be given away to the public). Commercialization of research ("tech transfer") involves additional costs and risks that are taken on by research institutions, researchers, and private entities. This PR announcement was likely not coincidentally following the release of the app to the public. There was likely additional costs outside of the original scope of research to make the app robust for public usage outside of the experiment setting.

Personally, I'm disappointed that something that sounds promising may not have a chance to stand on its own as an example of a viable application when other "brain training" apps have shown their more akin to placebos rather.

    [1] https://www.statnews.com/2016/01/05/brain-training-lumosity/

There's no reason to believe that this new app is any less snake oil than Lumosity

Its story has the same trajectory

I agree. Summary of findings:

“Test subjects who spent hours practicing [insert proprietary game here] scored better when tested on different games that require similar skills.”

Smells like a poorly designed experiment salted with commercial interests.

Yup. A similar thing happened with a training/game called the "N-back" test that allegedly could improve executive function and IQ!

Turned out the test of IQ was Raven's Progressive Matrices and it's never been decisively validated

https://en.wikipedia.org/wiki/N-back#Use_in_tutoring_and_reh...

https://hn.algolia.com/?query=n-back&sort=byPopularity&prefi...

Dual N-back is reported to improve working memory, which can help improve IQ. A excellent report is here https://www.gwern.net/DNB-FAQ

That's great except Dual N back is, at least for me, impossibly hard to play. And I have in my estimation a good working memory that enables me to work on multiple abstraction levels simultaneously without losing track.

Are there any tips on how to not miserably fail?

That's vague. Do you have specific examples of flaws in methodology or analysis in the published peer reviewed results here [1]?

[1] https://www.frontiersin.org/articles/10.3389/fnbeh.2019.0000...

Very cool analogy about the beakers and equipment. Something I never considered.

>>> The game involves asking players to watch a series of digits from two to nine flashing up one by one, at a rate of 100 digits per minute. Over the course of five minutes, players must press a button when they start to see a sequence emerge.

C'mon, this could be cloned in a weekend ;)

Publish neuroscience findings in a public access journal by all means. Even including pseudocode of the sequence generation algorithm.

But I am all for distributing their particular implementation via private sector partnerships. That could yield further funding and experimentation. And possibly alleviate taxpayer research dependency for their lab in future.

The work itself taps into some novel neuroscience. Do we we possess a Bayesian brain that estimates probabilities in real time? Altering a belief net based on new evidence. Or are patterns hard wired and must be learned. I think this sort of training game based on integer series could work just as well with text, images, music, video, animation, etc.

> Do we possess a Bayesian brain that estimates probabilities in real time?

You should read Daniel Kahneman's "Thinking Fast and Slow" - the answer is a resounding "No".

You might wish to read Gerd Gigerenzer as well as Keith E. Stanovich, Richard F. West & Maggie E. Toplak before citing Kahneman so quickly, and doubly so in support of such a tremendously strong claim as the one you just made.

For Gigerenzer, you might wish to start out here:

https://en.wikipedia.org/wiki/Gerd_Gigerenzer#Heuristics

https://www.tandfonline.com/doi/abs/10.1080/1479277914300003... (Available here: http://library.mpib-berlin.mpg.de/ft/gg/gg_how_1991.pdf)

And for Stanovich et al., you might want to start out here:

http://cognet.mit.edu/book/rationality-quotient

https://en.wikipedia.org/wiki/Dysrationalia

Stanovich et al. showed that Kahneman & Tversky's classical "System 1/System 2" dual process model as one too simplistic, outlining at least 3 systems involved:

* The Autonomous Mind

* The Reflective Mind

* The Algorithmic Mind

For points addressed by both Gigerenzer as well as Stanovich et al, see here:

https://en.wikipedia.org/wiki/Great_Rationality_Debate

None of this of course directly addresses your "Bayesian brain" point, however, for that, you might want to take a look at "some" articles by psychiatrist Scott Alexander, many of which speculative but citing a lot of sources you might wish to look into:

https://www.lesswrong.com/posts/ZiQqsgGX6a42Sfpii/the-apolog...

https://www.lesswrong.com/posts/euJm4RwkAptZnP89i/bayes-for-...

https://slatestarcodex.com/2015/08/28/mysticism-and-pattern-...

http://slatestarcodex.com/2016/09/12/its-bayes-all-the-way-u...

http://slatestarcodex.com/2017/06/28/why-are-transgender-peo...

http://slatestarcodex.com/2018/12/11/diametrical-model-of-au...

https://slatestarcodex.com/2019/01/10/paradigms-all-the-way-...

(And here, as a bonus, an article on dreaming, by Eliezer Yudkowsky: https://www.lesswrong.com/posts/8z2Fm2yaHpQz8rr5B/dreams-wit...)

Real time is also a funny concept when it comes to perception. There is a TED talk where the speaker explains how your perception of "now" is constructed over about half a second after receiving input. And can be retroactively overwritten in the next few seconds, if more convincing input contradicts the created picture. 100 images per seconds seems to be just a notch below this mattering, but probably sustaining this needs concentration.

Research I have seen suggests we do construct Bayesian prices but we don’t apply them rationally. If people assess that something happens 60% of the time, then in a series of 10 observations they will predict it happening only 6 times.

That is to say, they will bet on the 40% probability outcome 4 times, even though they believe the 60% is more likely each time

It took me a second to realize the meaning of this; do you have any links - sounds fascinating :)

Could be due to userbase size but, I liked the assessment and decided to purchase one month of pro to see what else was involved. Once I bought pro I did another problem solving focused workout. My percentile on the assessment for this category was 53% before pro now that I'm pro and did one 'workout' It says I'm at 89%... I find that hard to believe.

Edit: Upon further inspection it looks like the percentiles I was reading were for the specific game. It makes sense that for the pay-walled games there aren't as many people playing so it's not a representative sample.

Research funding in the US oddly doesn't seem to provide any rights to the IP to the taxpayers and only limited rights to the government, at least when it comes to the NSF and NIH (DoD research does seem to be different).

Disclaimer: I'm a researcher in the US.

https://en.m.wikipedia.org/wiki/Bayh–Dole_Act

Is what gives universities IP rights and allows them to keep research closed. Top Five Fascist laws in the books.

Fascist seems like a pretty extreme overreaction.

Corporatized public institutions are clearly a fascist move. Same goes for the public private partnerships in common wealth countries.

So anyone who implemented a PPI in the MHS is a fascist? Great way to devalue the term.

You are putting words in my mouth, this isn't what I said.

I called Bayh-Dole act a fascist law, because it encourages the privatization of public research. It is literally a gift from the public to the corporation and encourages public organizations to participate in corporate behavior which is antithetical to their actual stated missions.

The PPP (public private partnerships) in common wealth countries are aimed at achieving

* less oversight of government activity and accountability

* union busting

under the guise of efficiency and cost reductions. And are anti-democratic. Fascism is the description we give to a certain type of corporate-statist behavior. The wonderful (facetious) effect this has is that is imperceptible to a lot of folks while slowly shifting the ideology to the right.

I am not calling people national socialists. I never called a person a fascist. Bayh-Dole was probably well intentioned but the result has been the exclusion of public research for the public good. It stifles more innovation than it encourages. What term would you use?

I've always wondered why grant funded projects are allowed to be patented, do you think just preventing the IP is enough?

I've seen Mariana Mazzucato suggest the government as VC in that they get a portion of the profits from their grant projects, but then I wonder how they could safely regulate their own funding revenue?

I'm no expert in the area of research IP and govt funding, but my interpretation so far has been that the government provides the funding with the expectation that the research will ultimately improve society (even if it is through a for-profit business). It also values the training of graduate students with the funding.

But the opposite happens because the same scientists own IP and run the study to determine its worth. So we know less and waste more than having done nothing.

Allowing the university to get the IP is another way to fund projects. If the university didn't get any of the IP, grants would have to be larger to fund all costs.

Considering the existence of crown copyright, I'm surprised you would expect anything from the UK gov to be open source or similar.

One of my favorite things about the US is that works of the Federal government are public domain. It speaks to the fact the government is representative of the people themselves, not a particular person.

Now granted, universities don't have to release their publicly funded work either in the US, but I think that should be changed.

This goes a little off-topic, but you might be pleasantly surprised about Crown Copyright.

Crown Copyright doesn't deal with university research, only with works done by the government. Pretty much everything released by central government is now licensed through the UK Government Licensing Framework under the Open Government License (OGL), which is essentially CC-BY.

See http://www.nationalarchives.gov.uk/doc/open-government-licen...

Examples of loads of photos and documents that Wikimedia have taken copies of (and often reusing) under the OGL: https://commons.wikimedia.org/wiki/Category:OGL

That's good to know! I guess I was a little out of date, I mostly saw this come up when there was a big push to get stuff on en.wikipedia moved to commons, and all of the UK/AU/CA content couldn't be moved for obvious reasons.

Canada has their own version of the OGL too. Dunno about Australia.

https://open.canada.ca/en/open-government-licence-canada

One need not have an expectation of a good outcome to be disappointed that it doesn't happen.

Just because the government makes an investment in some research doesn't mean that it has bought the public rights to all commercial applications of that research.

This should be obvious on HN of all places. The government is just an investor in this context. It can demand whatever terms it wants, but the terms you're proposing are basically "100% equity in return for funding initial development costs." Nobody with options would take that deal.

The flipside here is that we're talking about a highly experimental technology that is not trivial to evaluate for consumers, and even likely snake oil (see luminosity)

So the argument to "nobody would take that deal" would be: nobody would make that investment. Lots of tax payer and government money goes to projects that would otherwise not exist, as such their rights and access to information should go beyond what a usual private investor would expect.

This obviously extends to academic freedom as well. If we're going down the road of treating the government as an investor in a 'university entreprise', then the government should obviously have shareholder control over a public university.

> Nobody with options would take that deal.

There are more than enough researchers and research which have no options, but are still valuable for society. Let the others be done commercially, more public money for those that really need it.

Definitely. Public funding should come with proper licensing restrictions that force open source.

Much publicly funded research never leaves academia and governments want an ROI in terms of seeing the research in products. US research agencies have been trying to incentivize commercialization of research and for PhD students to pursue entrepreneurship. They launched the NSF i-corps program to facilitate that, with the intial curriculum by Steve Blank.

Of course, this article is talking about UK research, but I assume the same desire to have research have impact applies.

Isn't that a little like saying that the government expects to see a "ROI" from educating children?

The positive effects of open sourcing science could be seen down the road and not in a direct way, but in more indirect ways.

Governments do expect an ROI from educating children. Educated children make employable adults who make taxable income which makes governments happy.

Researchers aren't (necessarily) philanthropists. They and the university both get a cut of the IP they develop to encourage them to both invent useful things and to get those useful things into production. If the commercially-viable fruits of this research had to be open-sourced it would be some half-commented github repo that no one would look at twice, because none of the researchers would have any incentive to make it anything more than that.

Thanks for Peak Brain that app is an ad hell hole.

Reply on news.ycombinator.com