Hacker News Reader

Comment by meri_dian | Hacker News Reader

The reality is that genuinely interesting, productive and novel results are hard to produce. Even for teams of highly intelligent researchers. The scientific industry is built on the assumption that such results are in greater supply - or more accessible - than they actually are.

View on HN

I don't think I could have put it better than that. Very well done. I am reminded of a parallel quote by Warren Buffett regarding the lack of truly good investments:

"I could improve your ultimate financial welfare by giving you a ticket with only twenty slots in it so that you had twenty punches - representing all the investments that you got to make in a lifetime. And once you'd punched through the card, you couldn't make any more investments at all. Under those rules, you'd really think carefully about what you did, and you'd be forced to load up on what you'd really thought about. So you'd do so much better."

I'm really surprised he said that. It flies in the face of diversified risk and everything it affects, like Sharpe ratios.

Buffet is not widely recognized as a fan of any form of the efficient market hypothesis in general or of diversification-reduces-risk specifically.

https://www8.gsb.columbia.edu/articles/columbia-business/sup...

"The way to become rich is to put all your eggs in one basket and then watch that basket." - Andrew Carnegie

The way Buffett invests is by reading all day, everyday and once in a while making a very informed investment. Most people haven't even reached the level of staying away from 'sure things' and panic selling during bad news.

How many people would've been able to stay away from IT stocks during the dotcom boom, to even not invest in the company of one of your friends. (Bill Gates and Buffett have known each other since '91)

How often do you, as a 'master', break rules that you would tell a 'apprentice' are immutable?

The common numbers thrown around are that with 20 stocks your portfolio risk is reduced by 70%. The corresponding ratios follow. So in that sense it's not that bad.

Of course that's just one form of risk. The less attractive aspect of a 20 stock strategy comes from the fact that the majority of the market's returns come from very few stocks - The 80/20 rule applies pretty well here. With only 20 stocks you'll probably miss out on the few winners that contribute all the market's gains. It's very easy to end up with a 20 stock portfolio with low risk/variance and low returns.

Of course if you are Buffet, then your goal is to pick 20 stocks that all outperform.

It doesn't mean you can't buy indexes or other diversified assets but rather that you should stick with what you choose and not try to time the market.

I think it's meant to be taken as the number of times you get to change your portfolio.

I don't think so. Buffett is more of a buy to hold forever kind of investor.

One way of thinking about this: Every time you add a new instrument to your portfolio you risk goes down. But return also goes down because there are only a very small number of excellent investments available.

This is a cultural holdover from the late 19th and earlier 20th century when novel results were much more plentiful. But now nearly all of the low-lying scientific fruit has been picked so the scientific endeavor needs to adjust its expectations. Unfortunately, culture changes more slowly than science.

BTW, the lack of new results is actually a good thing. It means we've got a lot of stuff figured out.

> But now nearly all of the low-lying scientific fruit has been picked so the scientific endeavor needs to adjust its expectations.

Of our current energy/resolution paradigms

We know that ~20% of the universe is Dark Matter and that ~75% is Dark Energy. All we really know about Dark Matter is that it falls down and doesn't really interact with itself or anything else, really.

Dark Energy? Yeah, whatever it is, it pushes galaxies apart, that's about all we got.

There are a LOT of pretty easy discoveries to be made yet, but we just don't have the engineering to make them yet. The LHC was a marvel of engineering, but it can't even come close to the energy required to produce (theorized) Dark Matter particles. If you tried to make a bigger one, you'd need an LHC with a radius larger than the Earth's. That's not going to happen anytime soon, not by a long shot. We need another method than particle colliders, probably a very large space-based telescope to stare at black-holes with jets.

Biology is also rife with 'easy' things to do, but ones that we just don't have the tech for yet. Strangely, with all of the data that the Googs and the Zuck have, we have not seen a similar paradigm shift in psychology, criminology, or economics. At least in public literature.

I didn't say there was nothing left to be discovered, I said that most of the low-lying fruit has been picked. Obviously there's a lot left to be discovered, but what's left is manifestly more difficult to figure out than what has already been figured out.

Food for thought: it could be that the already-picked "low-lying fruit" is responsible for new discoveries being harder to make. Once you get into a certain mode of thinking, or come to view the world a certain way, it becomes harder to see the world as operating any differently.

So, maybe it's not so much that we have stuff "figured out", as it is that we have mental models that, while they work well enough to make certain predictions, also constrain our thinking in terms of what phenomena are possible. The "EM drive" controversy is one example of this tension.

Very unlikely. We are where we are today because along the way there were a LOT of alternative theories considered and discarded, either because they did not agree with experiment, or because they lacked explanatory power. Science is actually very open to new ideas. We even know for certain that the current foundations WILL change (because dark matter, etc.) It's just that making progress is really hard because our current theories already account for ALL of the phenomena that are easily accessible here on earth.

>> because our current theories already account for ALL of the phenomena that are easily accessible here on earth.

I'm not so sure of this. Have you heard of Edward Leedskalnin, or his "Perpetual Motion Holder"? The textbooks that I have discussing magnetism do not explain this effect, which you can see an example of here: https://www.youtube.com/watch?v=eWSAcMoxITw

If you can explain this effect with our "current theories", I would love to hear it!

That's a neat trick, but I see no reason to believe that it's anything other than straightforward electro-magnetism.

It is a neat trick! Somehow those pieces of metal are storing a magnetic field like a permanent magnet, but release their "charge" when the field is broken.

There's another interesting effect where an electric charge can be pulled from that set-up. Lasersaber demonstrates that here: https://www.youtube.com/watch?v=S_ssUTRbRRs

I certainly can't explain it, and I don't think we have an adequate theory that does explain it (yet).

It's altogether possible that you're correct, but I'd say that most of the responsibility lies in this: if you look at pondwater with your naked eye, you can see cloudy pond water. If you look with a carefully made glass bead, you can see bacteria and protists, and thereby make a TREMENDOUS discovery: unicellular organisms. But if you want to get even CLOSER and understand more about what parts make up the organisms and other things at greater magnification levels, you're going to need a bigger, better microscope, and that will cost more money, require more care and understanding to correctly operate, and be able to scan less total area at a time.

In other words, much of the problem we face in the current scientific age is due to the fact that our instruments are now electron microscopes and Hadron Colliders, instead of carefully made glass beads and simple mirrors.

It's similar to problem faced in software of managing increasing complexity. No silver bullets known of yet for all those problems, just chronic application of powerful minds to the problem. :)

On the other hand certain technological leaps can create a whole field of new low hanging fruits.

For example within medicine when computer power and AI is advanced enough to help us much more connecting the dots between DNA, viruses, environment etc.

Yes, that's a very good point.

Don't make things up.

https://en.wikipedia.org/wiki/Superconducting_Super_Collider

Yes, but that collider can't get to the energy levels needed.

edit: My original post was waaaay too agro, I was way out of line. I misinterpreted your original post as meaning -the minimum size of a larger collider than the LHC would be larger than the Earth's radius-, instead of -a collider larger than the LHC that could hypothetically produce dark matter would be larger than the Earth's radius-.

for posterity: That collider can't get to the energy levels needed because it doesn't exist. It doesn't exist because it's construction was cancelled after the project was defunded. Where it to have finished construction, it would accelerate protons up to 20 TeV. The LHC reaches 6.5 TeV. All of that information is clearly stated in the link I posted and is completely uncontroversial.

Stop bullshitting.

> BTW, the lack of new results is actually a good thing. It means we've got a lot of stuff figured out.

Not necessarily. If anything, the more we find out, the more we realise how much we don't know. Take quantum physics, that stuff slowed down since the 1930s.

It's not a bad thing, the journey's always fun too. But let's not kid ourselves that we're close to figuring everything out.

Personally, I interpret GP as we have already proven many of the true hypotheses that can be simply and easily formulated, based on the categories and concepts that we currently find intuitive.

And this shouldn't really be a surprise. Our cultures and societies have had thousands of years of evolution; our human instincts have had at least hundreds of thousands of years. In that time, both culture and instinct have picked up many true facts about the world that have helped us survive. A lot of science has been taking those facts and systematically investigating how far they hold.

So these both can be true: the more questions we ask, the more questions we have to answer, but also that many of the 'easy' hypotheses have been mined out, and any true advance in science will by necessity pull us further and further from the realm where our basic instincts and intuitions can usefully guide us.

Quantum mechanics being the prototypical example, of course.

> the more questions we ask, the more questions we have to answer, but also that many of the 'easy' hypotheses have been mined out.

Like a tree? The main branches are the most visible, and so the most tackled. The smaller branches are only realised when you're deep enough in it. That's where our age's advantage comes in, as we have many more qualified 'explorers' than before, but it also makes it all the more important that the foundational branches are solid.

But then again it may be closer to readjusting our lens. The big questions (of the natural kind, not innovation like AI) today still pertain what we take for granted, like quantum mechanics is making us question the nature of reality again. But much like the discovery of waves once threatened the credibility of light as particles, I'm optimistic that reconciliation can happen, and to ascend to another level ... but where can we stop? When can we say "This is it"?

Just musing..

My father used to illustrate the progress of science as an island in the Sea of Unknown. The more you expand the island by gaining new knowledge, the more you get into contact with the unknown.

But the things we realize we don't know are much harder to figure out, they take more resources (time, money, study participants, analysis, etc). That's the reason things are, and should be, slowing down.

We are, clearly, close to figuring out most of the easy stuff. For our current definition of "easy" based on the amount of work, computing power and strength of the mathematical tools that we consider normal.

When you can do multiple experiments and predict the results to 8+ digits of accuracy there is little room for improvement. Your theory also needs to predict the same 8+ digits for all existing experiments while being either simpler or allowing for some new prediction that's also accurate and in disagreement with existing theory.

There really are very few edge cases left for QM 2.0

> This is a cultural holdover from the late 19th and earlier 20th century when novel results were much more plentiful.

I'm not sure about that. As a former working scientist, I would say that the primary driver of this is the promotion and funding structure of science. I do agree thta a lot of low-lying fruit has been picked.

> the primary driver of this is the promotion and funding structure of science

Well, yeah, obviously. I'm saying that structure is there because it's a holdover from previous generations where novel results were easier to obtain.

The funny thing is the results where gained either through paced pondering or by ridiculously talented individuals like von Neuman. Big science for non-applied fields is an oxymoron. Once a principle is figured out you can chuck a ton of people at it. But before that arrives, you can't pretend everyone is von Neuman and put them racing against each others. It seems there is no room for time and craftmanship in the publish-or-perish approach to science.

Have you read this?

http://www.thenewatlantis.com/publications/saving-science

If we are to measure science by its technological results, then we would judge Arrhenius' paper on global warming to be a failure because it didn't result in any iPhones being sold. The author illustrates a difference in value systems, but it's sort of inherent to those that one can't meaningfully argue one against the other. I think the problem is with our incentives, not our institutions.

No, I haven't. Thanks for the link.

or it could mean we're stuck in a local minima due to structural barriers

What constitutes the "low-hanging fruit" changes constantly. I suspect that trying to quantify them at any given point in time is as infeasible as it is useless.

Then all the more reason for replication. Shouldn't we focus on making the foundation as robust as possible, before building more on it? It's possible to discover something exciting, or completely novel, but "exciting" and "novel" shouldn't really be the sole pursuit of a scientist. Rather, all outcomes including 'boring' ones should be considered with equal attention.

Hard to produce is not the same as hard to reproduce.

Genuinely new results require developing new insights, new techniques or just making new guesses that happen to be right. All are rare enough. But once that is done, the information gets out and the experiments should be easily reproducible.

Some dispciplines get a lot of reproduction as a side effect of using last year's breakthrough research as "mere" technology for implementing next year's expriment. But I guess that sort of thing is less natural pyschology and sociology.

For many experiments, the cost (required effort) of reproducing them is the same as producing them, simply because you need to do all the same major activities once more. If they needed to spend an hour each with a hundred subjects, you need to do that too. If they needed to carefully run a long process on some materials, you need to do that to. If they needed to build a particular apparatus specific to that experiment, then you need to build or buy one.

And it's possibly more costly than the original since the initial lab likely was able to re-use some of their existing resources, trained people, infrastructure, lab animals, equipment (whatever matters for the discipline) which you don't have; they likely chose this particular experiment because it fits well with the stuff that they have that's not available everywhere. If they developed a particular experimental setup / gear / network of volunteers doing interviews in the subpopulation you're studying / cell or animal line / etc, then they're using that for many publications over the years, but you need to spend the whole cost to reproduce any one of them.

Good, point.

I was once an experimental physicist and never psychologist. In physics there is a lot of trial and error before you can get the experiment to do anything at all. Anyone who reads your paper needs fewer trials.

So I was going to agree with you that in fields where experiments work differently, reproduction can cost as much as the original study. But on second thought, that's wrong: and indeed it is the whole point of the thread.

There is always trial-and-error. You simply don't know if your theory is correct. And there is always risk of a methodolical weakness that you only understand after seeing the data. The trouble is after you have already spent all your time and grant money on your study, you have every incentive to paper over these issues. Indeed even if you are scrupulously honest, you will have to shove study in the proverbial file drawer - because you don't have the resources to try another iteration.

So I think it's still true that a new experiment costs more than reproduction, and many of the problems with studies arise from trying to pretend that cost doesn't exist.

The difference is that with a reproduction you know you are expected to succeed.

It's hard to find a Roman gold coin in farmer's field but once you've found your first one you can be more confident that the second will be easier to come by.

Meanwhile, false positives are as easy as ever to come across.

Easier, if you factor in how much easier P hacking is with modern software.

"Everything that can be invented has already been invented" - Charles H. Duell, 1899.

We don't know anywhere near "most" things. The supply of novel discovery is not the limitation.

"Everything that can be invented has already been invented" - Charles H. Duell, 1899.

Please don't use that quote. It's apocryphal; he almost certainly never said that.

It's not about how close we are to "knowing most things," but about how accessible new learning is. In a game of musical chairs, all the easy chairs are taken.

How about this quote from Lord Kelvin:

> The beauty and clearness of the dynamical theory, which asserts heat and light to be modes of motion, is at present obscured by two clouds. I. The first came into existence with the undulatory theory of light, and was dealt with by Fresnel and Dr. Thomas Young; it involved the question, how could the earth move through an elastic solid, such as essentially is the luminiferous ether? II. The second is the Maxwell–Boltzmann doctrine regarding the partition of energy.[0]

As I understand it, the two clouds were dealt with via special relativity and quantum mechanics.

[0] https://books.google.com/books?id=YvoAAAAAYAAJ&pg=PA363#v=on...

I think he used that quote with the year timestamp to show how silly that quote is.

Yes, and david927 is making the counterpoint that the quote is not silly now, nor was it silly then (because it was not made then).

Or to be exact, it's not that the quote isn't silly but that it's not a quote.

The average age of a Nobel laureate continues to climb at a steady rate. We are reaching asymptotic knowledge gains in all the major fields. This fact combined with the tenure track / novelty factor leads to what we see today.

There is, of course, an alternative explanation.

Our society has changed, dramatically, in the past 30 years. We've gone from a completely disconnected society with connections spanning arm's reach to a globally connected society with connections spanning just as far. Baby sitters have been replaced with digital tablets providing endless entertainment and ensuring no person, adult or child, ever need sit with their own recreation being solely within their mind. One needs only look at the slew of digital devices whipped out at any eatery, lest individuals be left to their own thoughts for an agonizing 10 minutes.

The internet and our technology has shattered nearly all educational barriers. Getting an MIT caliber education for free is now something any child in a developing country who has access to the internet can do, if so inclined. On the other hand they can find countless entertaining drama to read or participate in, find an endless supply of porn, engage in social media and other online sites that are specifically designed for addiction, or find an array of games and other forms of fast response stimuli all designed to make him or her feel good.

Einstein constantly related his tales of meandering strolls as he pondered his work. Another individual that has managed to achieve some great things in modern science is Stephen Hawking. He's mentioned spending decades thinking intently, even if not exclusively, on singular topics. And there is absolutely every reason to believe that is in no way bluster. Given his condition, his mind is his recreation.

Is the breaking down of information barriers in our society helping more than the massive commercialization and directed addiction towards all sorts of mediums and vices along with the rapid reward systems they offer, is hurting it? It's not just high level academia that's failing. Education systems to be deteriorating at all levels with US (which is the center of much of this change) scores in math and science rapidly falling behind much of the rest of the world. Healthfulness has fallen to dangerously low levels. And more. Regression is not impossible.

This explanation has no support in actual evidence whatsoever.

Finland has seen a clear decline in literacy after the emergence of smartphones in the last decade (every child past 7 has one).

Although, to be fair, the direct effect is explained by increased inequality:

http://www.eera-ecer.de/ecer-programmes/conference/20/contri...

Yet, I know how digital shit affects me. I'm not talking about the devices themselves, I'm talking about being flung up by some stupid pointless shit in social media or by the colorfull game which totally hijacks the reward mechanisms in my brain.

I totally fail to see how the effects which promote shallow, fast reward seeking cognition are not not detrimental... yeah yeah, it sounds like greeks whining about reading spoiling the mind but the difference is that our media is engineered to distact and hijack the mind for no valuable outcome whatsoever.

Sure, there are lots of positive effects but that does not mean one should not discuss the negative ones as well.

And no evidence against it. I believe it was intended as a hypothesis.

While most of the comment is indeed a very detailed and interesting analysis, the last part is unfortunately looking like a "it was better before" nostalgia bias.

I'm currently writing my phd project (after having worked for 10 years in a company) and I'm absolutely stunned by the current easiness to access papers and informations compared to just 10 year ago when I left university.

I concure however that a new and real challenge is now to be able to filter the noise induced by this overwhelming volume of available information. But I don't think nostalgia is an useful tool to reach this goal.

PS: To illustrate https://xkcd.com/1601/

(And I believe there was another comic that it traced it back to Greek philosophers that rejected writing. Thanks to whoever remember the link!)

The US is a net importer of smart people... that may change though.

Could you say the expectation of novel discovery is to blame, though? If it becomes harder to blaze new trails, but the expectation of discovery remains the same, then scientists who wish to keep getting recognition and pay must find ways to fabricate discoveries to keep up.

I would say the expectation that novel discovery is easy could be to blame. Also that the incentives discourage the process from working in a way that would allow further novel discoveries. Science needs per review and replication of results to work properly. If the incentives discourage this then novel discovery gets harder. Scientific discovery is a long-term game which means business methods are poorly suited to managing it.

As an aside, that quote is apocryphal, by the way. The actual Charles H Duell said almost the exact opposite

Reply on news.ycombinator.com