Take neural nets. FLOPS/$ crosses some magic number so that universities can afford the hardware, and bam; AI everywhere. No obvious theoretical breakthroughs since the 70s, but suddenly the field gets intense due to economics.
Evolution is more debatable, it seems that Darwin's voyages of discovery were certainly an important part of his thinking. Still, it happened ~200 years after European ocean expeditions really started taking off, so doesn't seem like it was cutting edge experimentation.
The evolution of ideas seems like a much more plausible explanation.
Microscopes were invented in the 1600s and cell biology as a discipline began soon after. But the dynamics of speciation were perhaps too grand to have been noted in the minutiae of slides.
And it would not be for almost another century after Darwin till we understood that the mechanics of evolution were driven by random mutations of DNA.
Leibniz I'm not sure about, but given the timing? There was definitely a lot of interest in basic physics at the time and I bet that was related to his interests, somehow. It isn't a hard set of coincidences to imagine. That was all being fueled by improvements in measuring technology.
Evolution of ideas is plausible, but it suggests a remarkably homogeneous intellectual environment in a time where communication was expensive. Economic forces are a lot more likely to me, they scale. What we see in maths is usually one genius can discover a huge amount of stuff very quickly, they don't need to wait for other people to have ideas for them. They just need a little push in what to direct their attention to.
> Evolution of ideas is plausible, but it suggests a remarkably homogeneous intellectual environment in a time where communication was expensive.
Science doesn't often advance alone, it requires communication between scientists. And that same communication spreads ideas among those scientists, until they build a common understanding.
It is impossible to do physics by thought experiments. There are an infinite number of logically consistent mathematical models of universes out there, only a small subset of them match up reasonably well to the one we exist in. Science requires experiment. It doesn't have to be the same person doing the experimenter and interpreting the results, true enough, but the interpreter is still bottlenecked by data collection.
> Science doesn't often advance alone, it requires communication between scientists
No it doesn't. The whole point of this thread is that science often advances in parallel with no communication between scientists. Once the raw data has been collected - which is an economic issue - or conditions arise where a new problem needs to be solved - also economics - then it is a relatively short matter of time until someone comes up with a compelling theory. One very bright scientist can often exhaust a big chunk of what can be deduced from the currently collected data. And the implication of this is every time there is an economic advance in the cost of data collection there will be a big boom in scientific understanding.
People rediscover calculus to this day, every so often we all get a laugh because some very bright doctor figures out he can sum rectangles under a curve. The only limiting factor is whether people realise that they need to describe curves, areas and tangents precisely.
I can only suppose that you are taking 'communication' to mean only direct communication via face-to face conversation or the exchange of letters, but communication is a transitive operation, and, in fact, your thesis is tacitly predicated on this leading to the dissemination of knowledge. The more I learn about the actual history of science (as opposed to 'scientific genius' hagiography) the clearer this becomes.
If you are looking for an economic perspective on this, the emergence of the infrastructure of academic publication would be a good place to start.
Thought experiments are an important tool used by many brilliant physicists when it is not possible to carry an actual experiment yet. Their usefulness is in the opposite of what you say: sure, they can't prove a teory correct, but they can point out potential problems by finding contradictions. As an example, here you can find a famous and very important thought experiment in the field of thermodynamics: https://en.m.wikipedia.org/wiki/Maxwell%27s_demon
Strongly disagree. The mathematical universes are heavily ordered by their simplicity and logical consistency and you can sort through them with thought alone.
Also, anyway, the act of 'doing physics' includes a lot more than the somewhat small category of 'testing hypotheses with experiment'. It's like 95% model building and 5% experiment.
Sure, but for the vast majority of physical theories before electrodynamics, our senses plus a few ancient instruments (meter sticks, hourglass, scales) were plenty enough. Galileo's relativity principle and his observation that all objects fall at the same rate in a vacuum were based on thought experiments and physical intuition, not actual experiment (apparently, he did try later to actually perform the experiment, but I think it didn't really work).
> The whole point of this thread is that science often advances in parallel with no communication between scientists. Once the raw data has been collected - which is an economic issue - or conditions arise where a new problem needs to be solved - also economics - then it is a relatively short matter of time until someone comes up with a compelling theory.
Sure, people can discover things in isolation. But science only advances when others learn of those things and can build upon them. Neither Newton nor Galileo nor the person in the article invented all of their theories from first principles - they were building on lots of thought and ideas that they had learned from other scientists.
> People rediscover calculus to this day, every so often we all get a laugh because some very bright doctor figures out he can sum rectangles under a curve.
Actually, calculating surface areas in this way is very very old - according to this Science article [0], it was already in use around 50 BCE. So it's not actually surprising that it has been rediscovered independently. I don't think much of Newton's work has been, apart from Leibniz (though that might also be because it's been so widely disseminated afterwards).
[0] https://www.science.org/doi/full/10.1126/science.aad8085
Tell that to Einstein.
You can pretend the world is filled with tiny creatures all day long, but until you can see them with a microscope, it’s just an imaginative idea, not science.
1) Schmidhuber does NOT claim to have invented it. He even provides lots of really old references. You know it's old when he didn't invent it, at least in his own mind.
2) even with his generous attributions, "the first application of backpropagation to neural networks" is from 1980.
3) "LeCun et al. (1989) applied backpropagation to Fukushima’s convolutional architecture (1979)".
In other words, the chain rule is really old but figuring out how to use that to adjust weights in neural nets was surprisingly unobvious. It was even more unobvious that that was a good way of adjusting weights.
These results would have all started to happen at about the time the cost of computation was within reach of the researcher's budgets. The "theoretical" breakthroughs are of the form "we can implement this technique from the 60s and get good results". Which is impressive, but it does not represent breakthroughs of knowledge as much as incremental improvements in hardware crossing key thresholds. The breakthrough is detecting that hardware can make something work now.
It most certainly would. Not in the 50's, of course, but in the 60's and 70's.
http://www.roylongbottom.org.uk/whetstone.htm
Look at the MFLOPS columns.
It seems to me that we had to wait until decent memory sizes and decent fp performance was a lot cheaper and therefore much more accessible => much easier to do experiments without having to justify them to higher-ups => somebody figured out 1) how to do backpropagation on neural nets and 2) that it was useful.
In other words, it wasn't obvious at all. It required experimentation.
It would have been practically useful from the 60's (for small neural nets and high-value problems) and 70's (not so small neural nets or lower-value problems) if somebody had figured out how to do it and that it was a useful thing to do.
It's odd how blatantly simple stuff can be so difficult for even experts to see sometimes.