Hacker News Reader

Comment by ISL | Hacker News Reader

Submitted because I really wanted to comment on this sentence: "For a startup, work needs to be both faster and more rigorous than an academic lab."

In our academic lab, if we knew how to be faster, more rigorous, or both, we'd be doing it.

The notion that a startup can do rigorous research faster than academia is curious. The scientific journals are not full of the output of startups. Industry in general, and startups in particular, can't expend the resources on covering corner cases and tidying loose ends. The profit is generally in getting the gist of an idea, not in writing it down, vetting every detail, and sharing it broadly for free.

View on HN

I chuckled when I read that. Apart from the uninformed way this was expressed, the author's actual interpretation of this statement comes after that:

"In academia, in order to publish a paper, often you just have to get it to work one time out of ten – so you think, OK, I’ll just keep doing the experiment until it works. We need it to work nine or ten times out of ten."

So the author confuses it with engineering. Then -- allow me to be a bit cynic -- the actual meaning of the article would be something like:

"If you want to be a scientist at a startup, you need to become an engineer."

> "In our academic lab, if we knew how to be faster, more rigorous, or both, we'd be doing it."

There are some cases where it's obvious how to be faster (and occasionally more rigorous): judiciously throw money at the problem. In academia, where grant funding is pretty limited and equipment is expensive, you probably want to only have enough infrastructure for, say, the 50th-90th percentile of load (microscopes, thermocyclers, analysis compute power, what have you), and accept that 10-50% of the time there will be a queue. If the difference between being successful in the market and having your lunch eaten by someone else truly is a matter of weeks, then it makes sense to you and your investors to put some extra money into a widget that will sit idle for most of the time but that helps when things are crunched.

That said, money can't fix everything---9 women can't produce a baby in one month!

> 9 women can't produce a baby in one month!

> Then you're just not thinking hard enough!

- My boss

You could have 9 women on a constant baby producing staggered cycle so one baby gets produced every month. Then, when you need a baby you will be able to get one in 15 days, +/-15 days.

you would need 12 to be precise.

'Rigour' in the article means 'intensity'. 'Rigour' in science means 'validity'. A study with high rigour means that they've nailed down more loose ends than a study with low rigour. The two concepts are largely orthogonal - and it's disturbing that someone telling scientists what to do has used the wrong version of 'rigour' for that industry.

Ironically, using the 'rigour' of science, startups require much less rigour - startups just need something that works enough, not something that is as correct as can be done.

Academic rigor applied to industry would be a type of overfitting. Just as one example, customers don't care how accurate a fitness monitor is, as long as it is vaguely effective. Yet, few journals or funding agencies would buy your claim of building a fitness monitor unless you can rigorously demonstrate its performance and novelty.

Of course, the story is different in pharma and a few such fields. Theranos is an example of how lack of rigor and openness can damage healthcare startups.

> The scientific journals are not full of the output of startups.

Startup's goals are generally completely orthogonal to publishing.

Here is one answer in the form of an analogy -- but I preface this by saying that comparing the set of all academic science with the set of all industry science is fraught at best -- academic science is to cottage industry as industrial science is to factory production.

A related consideration is that training grad students and post-docs is a key component of most academic science. The requirements of training often limit the size of teams working on a single project with the PI-trainee relationship dominating the organizational structure.

As the "PI" of a science startup R&D team, I can start and stop new projects at will with varying team sizes and mandates without the consideration that my folks need to produce a body of published work to further their careers.

The author might have meant aiming for actual results instead of publishability. Academia, in general, is fucked up. Research is being done in order to publish - which means lack of real rigor and lot of fake one. Studies are being performed using bad methodologies, and then massaged and/or repeated until results cross the magic threshold, at which point they get published in 10 papers that say the same thing in slightly different words.

It's a terribly inefficient process that could be optimized by changing the goal from "publishing" to "making something that actually works". Thus you could achieve greater speed and more rigor at the same time.

(Basic research could probably be optimized too, although not through market incentives.)

While I generally agree with your comment, I think the one big advantage of private companies over academia is that they're typically much less resource constrained. Academic labs sometimes take huge shortcuts on cost with strange side effects. Often the way to be more rigorous or faster is "have fewer resource constraints" and even a mediocrely funded startup can have much more money than a well funded academic lab. On top of that, the more experienced team members spend a smaller percentage of their time fundraising.

The same thought occurred to me; I think it's probably more along the lines of "faster and relying on more intuition".

I don't think this is a new concept; Jon Gertner in his excellent book "The Idea Factory" writes how the researchers at Bell Labs switched from basic research to development during WW2 and operated in a similar way. If anything, the urgency of development in the war effort resembled a startup in how it accomplished within a few short years projects that would have taken decades in peacetime.

Or, faster and relying on translational or post-translational stage research.

If you're building a company, odds are you aren't doing fundamental research (and you might not even be doing translational research).

>"For a startup, work needs to be both sloppier and done more arrogantly than an academic lab."

Fixed that for you.

Reply on news.ycombinator.com