Hacker News Reader

While I agree with you that the rate of paradigm shift increases over time, I argue that their average impact/scope decreases. Newtonian physics opened more room for technological progress than quantum mechanics, etc. Earliest inventions have the most impact (the invention of the scientific method having the most impact of all?).

So my point is that if you look at paradigm shift rate not in volume but in impact, you'd get a linear rate.

For this reason, an AI given imagination and exploding resources would still only make linear progress.

I didn't notice your point about impact. Impact is hard to measure purely objectively, but I agree Newtonian physics, being more fundamental, has had more impact. But it's also had 325 years to do so, whereas Information Theory has had only 64 (to stay with these examples). Given an exponentially expanding search space, there's always more to see (but this is an assumption, not a conclusion; I think the nature of the search space is the essential question at issue here).

How much impact did Newton have on the world by 1751? How much impact will Shannon have had by 2337?

Transistors couldn't have been designed without understanding Quantum Mechanics. Bye, bye modern computers.

And quantum mechanics could not have been thought up without newtonian mechanics...

> Perhaps a mathematical model of computation might help, of computable functions and Kolomogorov/Solomonoff program length to compute them. Using this model, the search space of functions increases exponentially with program length - but at what rate would the number of "interesting" computable functions increase? And with what density (with respect to the search space, i.e. how hard they are to find)?

Like AIXI?

http://www.hutter1.net/ai/aixigentle.htm

"To use the given examples, Newtonian physics, quantum physics and Information Theory began in 1687, 1930 and 1948."

Some things that we consider significant today won't seem significant from the perspective of centuries. Similarly, someone in 1750 would be able to list significant new fields developed recently that we would omit. For example, Newton also did work in optics, which we pretty much forget about today.

While I would consider Newtionian physics to be much more fundamental, I would challenge the notion that it had the most impact. Compare the ~1900 feeling that almost everything had already been invended. Newtonian physics impact to technology was mainly mechanics. Then came thermodynamics which gave us (steam &) combustion engines and more. Nuclear & quantum physics lead to bombs, reactors and finally seminconductors and so on. I would say todays thnological relevance of thermodynamics and quantum physics is at least as high as that of Newtonian physics. Also: most of todays physicists do not look for Bosons, they do things like advancing semiconductor masking technology, liquid crystal physics and the like - i.e. they are not "fundamental" but "applied" physicists. Also - with respect to the original article - isn't the much faster exponential growth of the resources invested in science since around 1900 mostly an indicator of expected technological returns by society at large? Science is a huge economy success story since about a century ;-) The question then would be how long can we afford to sustain the current exponent?

Intelligence and Imagination could not be more separate.

I'm defining intelligence as sheer IQ, analogous to a computers' ability to process information. Imagination is so essentially human in nature that there is no way to describe current AI as "imaginative". For example, the person with the highest IQ in the world is not more creative than the best musician. Some of the smartest minds in the world cannot "think outside of the box" effectively, because their minds are not wired that way. Intelligence is not imagination.

Can anyone please explain how an intelligence-self-improving computer will shift from being incredibly smart to wildly imaginative?

That's the one hole in singularity that I can't figure out.

You started off defining intelligence as something limited and not independently useful, and then you concluded that its presence was limited and not independently useful.

The answer is that it's not actually obvious that "intelligence and imagination could not be more separate" or that "imagination is...essentially human in nature," so when most people talk about AI, they are not excluding those abilities.

Imagination isn't any more human specific, than inteligence, or walking upright. That we can't do machines that are imaginative now, doesn't meant it's impossible to do.

Besides - what if imagination is just heuristic to search solution space quicker? Maybe machine with orders of magnitude more processing power can give better solutions without using this heuristic? That would mean it's us who thinks inside the box (because we use this heuristic), and computers can search solution space exhautively, so they will come up with solutions that escape our heuristic - we wouldn't even consider such solutions.

Yes, it doesn't really matter how intelligence is defined, just that in aiming at creating an artificial mind, we'd try to include whatever was useful, including imagination. My reasoning is, if one machine can do it (us), it's possible for another machine to do it. (Though there's a question of whether we are smart enough to create that machine.)

Exponential search space is a killer. Some form of heuristic seems necessary. Perhaps we will one day be able to simulate the heuristics we ourselves use - and then improve on them (e.g. apply them to more data than we can; use more complex heuristics; etc along the lines you say). Given variation between intellectual abilities of human beings, it is does not have an obvious fixed limit, and we may be able to extend the range of that variability. Though I expect increasing the associations an "imagination" can find suffers from exponential explosion.