What happens when your startup gets debunked by science?
fastcompany.com
fastcompany.com
They may need to adjust their marketing to not run afoul of the FTC, but I'm sure a slight pivot will suffice. They will live or die by how well they satisfy a (psychological) consumer need, not whether the product "works." Do any of the exercise products advertised at 3am ... "work"? It doesn't matter.
[Note: this comment is intended as an unsnarky statement of fact.]
With any physiological system of sufficient complexity, whether it be human digestive/metabolic systems or the neurological systems responsible for memory and intelligence, there will be opportunities for profit as long as improvement of that system is desirable and a small fraction of the population believes they have the chance to improve it and the cost is reasonable. Plus, there will likely always be "success stories" to cite simply due to the placebo effect.
We have a massive alternative "medicine" industry that supports not only the people in it, but also, ironically, the people producing the constant flow of content criticizing alternative medicine. Lumosity will be fine.
People are remarkably good at ignoring things they don't like to believe. Other people are good at exploiting those people.
See, for a particularly bad example, "Brain Gym". This is a set of nonsense exercises that children are forced to do to increase mental alertness. It has some bizarre statements ('processed food does not contain water'; breathing through the nose gets oxygen to the brain faster, etc).
This nonsense is depressingly common in UK schools.
I think that was intentional, and that the answer is either "no one knows" or "caveat emptor". Or both.
I suppose the broader point I'm trying to make is they won't have to change as it is hardly like there's any scientifically literate mainstream media to hold them to account, and they've bought off the public media alternative.
A gradual decline in public interest in public television and radio. That, in turn, has been caused by the increasing diversity of offerings on cable and satellite TV and radio, compared to 30 years ago. There was a time when the only way to see anything resembling scientific or educational programming was to tune into public TV or radio. Not any more -- there are dozens of channels offering this kind of content now, of widely varying quality, but certainly competing with public TV.
As the number of advertisements and pledge drives on public TV has increased, the public has begun to see public TV as just another commercial-supported TV offering, nothing special. And guess what? They're right -- there's no difference.
Don't forget the conservatives in management who want to "diversity" the programming and defunding due to the "socialist" content.
Yes, very true. Management reacts to a conservative social trend by avoiding content that might be looked on as too liberal, this offends liberals, which causes a further drop in audience numbers, and a death spiral begins.
This speaks to a general rule in television programming -- everybody want to have the "least offensive program". The least offensive program is the program that is so bland that it gets the largest audience simply because it offended the smallest number of viewers.
I stopped listening partly because I was getting really offended by some of that stuff, and as stated above this brain science stuff really got on my wick. But also because the quality of the news in the morning was garbage. Firstly very little news, way too much human interest story. Secondly the reportage was pretty shoddy. Thirdly the analysis was facile, ill-informed and wrong way too often. Certainly not as bad as something like CNN, Fox etc. but not the quality they boast about. And when you can now get the BBC news anywhere in the world you can easily see what a poor product the NPR news has become.
I guess finally the talking heads are just so outdated. They represent a washington-insider-y worldview that has little or no bearing on reality. They're an aging media corps which still thinks things run the way they do at their cozy DC cocktail parties, genteel disagreements but we all get along. They have little to say that matches the bitter corrupt partisan nature of things in the US today. It's not that I want them to perpetuate that, but denying it exists helps no-one. Describing debates as though they're these deep seated points of principle and taking every statement at face value benefits no-one when it's easily determined, and apparent to anyone listening what's really going on.
IMNSHO
Why would you devote any pretense of research and the associated costs if your customers don't care? Investors don't care as long as the money is flowing in and our regulatory agencies are toothless to deal with "health" product scammers.
There's plenty of money in selling placebos to consumers.
They're really not any more empowered than they are with nutraceuticals. And yes, all health products should be regulated, and they shouldn't be as deregulated as they are now.
"If you ever want to sell b2b (which is where healthcare dollars are at) then you'll need more."
Non-evidence based medicine doesn't seem to be running into any roadblocks.
> Luminosity should be fighting alongside those meta-analyses and investors should be demanding proof points in terms of p-values.
How many $$$ is a p-value or meta-analysis worth?
> Fail (scientifically) fast.
That doesn't sound very profitable. How does that put more $$$ into the founders' pockets?
I'd say healthcare companies in the behavioral/digital/connected health segments that have scientific strategies are outliers. Without regulation it's easy to justify trying to sneak by... but hard to cross to the b2b market.
Different cognitive training programs are different - just like different anti-cancer molecules are different. Some are going to work, and some aren't.
[1] http://www.ncbi.nlm.nih.gov/pubmed/17046669 [2] http://www.ncbi.nlm.nih.gov/pubmed/16888038 [3] http://www.ncbi.nlm.nih.gov/pubmed/19220558 [4] http://www.ncbi.nlm.nih.gov/pubmed/22918524 [5] http://www.ncbi.nlm.nih.gov/pubmed/22579081 [6] http://www.ncbi.nlm.nih.gov/pubmed/19448187
Problems obvious to a scientist:
1. Poor controls. Both the experimental subjects and the researchers know, or can easily establish, which are experimental subjects and which are controls. A built-in bias.
2. The study was conducted by the company that stood to benefit from a positive result. An obvious procedural flaw.
3. Inadequate precautions against counting a placebo effect (an imaginary benefit) or Hawthorne effect (any attention paid to the subjects improves their state) as a meaningful outcomes correlated with the specific treatment rather then any treatment.
But I am sure you already know this -- it presents the usual litany of legitimate objections to this kind of study.
> Different cognitive training programs are different - just like different anti-cancer molecules are different. Some are going to work, and some aren't.
That's true but irrelevant. The only meaningful question is which methods work in principle. There are no methods that consistently work, and no one can explain the outcomes, only describe them. Which means it's not science.
1. The trials are blinded, with more advanced studies using active controls (e.g., ordinary computer games[6], adult education[1,2,3]). Study staff administering cognitive assessments are always blind, and participants are blind in studies with active controls. The human consent forms describe the studies as comparing two forms of cognitive training. This is a good blind - in one study the wife of a control participant called my to tell me her husband loved the cognitive training because the art history education made him go to the museum more with her :-)
2. Study [3] in my post was conducted by Mayo and USC (and funded by Posit - the money has to come from somewhere), studies [4],[5],[6] were conducted entirely by independent academic investigators.
3. Active control groups control for the placebo/Hawthorne effect.
As I see it, the only meaningful question is which methods work in practice - and the studies show us that. If what you mean is do we know what the mechanism of action is, we are speeding up information processing flow through the brain and making it more efficient - speaking loosely, we're improving signal/noise ratios. Reference [1] in my post has an extensive conceptual background to this approach. And we've seen that with studies of EEG/working memory [7]
But you guys were apparently unable to address Robert's questions: http://blog.sethroberts.net/2013/03/09/a-few-answers-from-po...
Easier said than done:
"Why Most Published Research Findings Are False" : http://www.plosmedicine.org/article/info:doi/10.1371/journal...
> As I see it, the only meaningful question is which methods work in practice ...
That's a dodge around any requirement for scientific evidence, because without rigorous science, anything "works in practice". Astrology, as just one example -- many members of the public would be surprised and offended to hear that astrology has no evidentiary support whatever.
When the studies you funded have been replicated by independent laboratories, then you will have crossed the threshold of science. Until then, it's not science, it's marketing -- especially when you think it's constructive to say, "the only meaningful question is which methods work in practice".
Because the two possible sources for such science are neuroscience and psychology. Neuroscience hasn't evolved far enough to offer any meaningful evidence for or against the claims, and psychology isn't even a science as the term is defined (a discipline that supports or discards its theories based on experimental evidence).
That doesn't stop outfits like Luminosity, in fact it helps their marketing to be able to offer hand-waving claims that can't be meaningfully refuted.
> From a company making bold claims like this I want to see a continual string of increasingly sophisticated trials, each one giving increasingly robust indications of success.
Why would they do that? The drug industry doesn't do that, and they have a much higher chance to conclusively prove or disprove the value of their products. But they know better.