Hacker News Reader

Comment by mindcrime | Hacker News Reader

then it hits a ceiling

Or apparent ceiling. Sometimes ideas haven't really "ceilinged out" so to speak, but they've run into an artificial barrier due to available compute resources at a specific point in time, etc.

Somebody (Geoffrey Hinton, Andrew Ng, I don't remember exactly) made the point that neural networks were in a pretty moribund place, until the confluence of three things came along: more compute power, more data, and algorithmic improvements. Given that, one wonders what other older ideas are waiting to be resurrected by a similar confluence.

View on HN

> artificial barrier due to compute resources

That's not really an artificial barrier. The program which encodes General AI can already be discovered by brute force, given sufficient compute resources.

Let's assume that evaluating a program takes one microsecond and one atom, and that we can parallelize the search across every atom in the observable universe. If the AGI program is 500 bits long, it will take about 10^57 years to find by brute-force search.

There's a difference between ideas for which there is not enough compute right now and ideas that are computationally intractable according to our knowledge of the physical universe.

Even more fun, I learned from Bruce Schneier's blog post [1] that it takes a minimum 4.4E-16 ergs of energy for an ideal computer running at 4.4K to "flip a bit" in an ideal setting.

Wolfram says there are roughly 2E76 ergs of mass-energy in the universe. [2]

So even if all matter and energy in the universe were put together to power a computer that did nothing but count upwards from zero, there's only enough matter and energy in our universe for the computer to count to about 10^91, or roughly 2^304 [3]. That's not even counting trying to use each resulting set of bits for anything.

So I don't think we'd be able to design too many useful computers by flipping bits until a useful algorithm comes out (at least not in this universe.)

[1] https://www.schneier.com/blog/archives/2009/09/the_doghouse_...

[2] http://www.wolframalpha.com/input/?i=mass-energy+in+the+know...

[3] http://www.wolframalpha.com/input/?i=log2(mass-energy+of+the...)

So, given the Shannon number, even the universe itself can't be used to brute force a game of chess.

Simulating evolution could also be called brute force. Doing that at a scale comparable to our own evolutionary history may be way out of range for present computers, but not foreseeable ones. (http://e-drexler.com/d/06/00/Nanosystems/ch1/chapter1_1.html) Likely we'll get there sooner in some not-quite-so-brutish way -- I'd hope so for ethical reasons -- but that way is available.

There is no reason to assume that evolution always brings about intelligent live. Anything "higher" on the ladder than bacteria could very well be an aberration that only arose due to very specific circumstances on prehistoric Earth.

I think we have more than a sample of one instance to look at though. The distinctive thing about humans is more that we operated collectively according to social rules, than pure intelligence.*

Due to evolution, the coordination of individual entities has happened repeatedly. Individual molecules became living cells, some suggest living cells merged into cells with mitochondria, viruses and cells merged, single cells merged into multicellular organisms, insects organized into social colonies, and now primates, i.e. humans have also organized into social colonies.

This doesn't prove evolution always creates social multicellular life, but it means (IMO) there is a clear pattern that provides a reason to assume it does short of specific reasoning or evidence about a barrier - why should the bacterial level be unique, and which bacteria would be the highest level?.

*I don't think you can separate intelligence from a social context. What we think of intelligence is closely related to things like the willingness to give up current rewards for long term ones. But it's not "smart" to do that unless you have justified faith that you live in a stable society where people are trusted to follow social rules.

There are a couple of most plausibly hard steps: the origin of life up to just before bacteria split from archaea, and the origin of eukaryotes, which happened just once after something like 2 billion years. But I had in mind something cheaper more like scaling up the idea of http://www.karlsims.com/evolved-virtual-creatures.html. Intelligence has evolved in animals as unrelated as octopi and us.

In an artificial setting you can manipulate the conditions to favour whatever you want.

FYI, we know what the theoretical physical bounds are:

https://en.wikipedia.org/wiki/Bekenstein_bound

The program which encodes General AI can already be discovered by brute force, given sufficient compute resources.

Even if that's true (and I don't think anybody can prove that point one way or the other just yet), who's to say that we have "sufficient compute resources" right now? That's what I'm getting at... an idea might be useful / valid in principle, but just not completely usable today because we don't have enough compute cycles available. The analogy is to ANN's which were limited by the computers available in the 80's and 90's, but became markedly more useful with modern CPU's and - even more so - with the advent of modern GPU's.

The point is that the barrier is REAL. It isn't "apparent" or "artificial". It is a fact about current computing resources.

the barrier is REAL. It isn't "apparent" or "artificial". It is a fact about current computing resources.

Right. And let me add this: what I was getting at is that the ceiling is not an innate part of the approach itself. Perhaps I should have said "external" instead of "artificial". Anyway, yes, the point is about current computing resources vs computing resources (and other factors) at some indeterminate point in the future.

How exactly? Do you have a codeable definition of General AI?

It's a truism and as such it doesn't really say anything meaningful. The limiting factor is "sufficient compute resources". Assuming that a general artificial intelligence is possible and can be coded as a program, then whatever that program is could be created by something as dumb as a random number generator.

Of course, given that it seems obvious that any program which encodes a general artificial intelligence will be complex, I expect it would take longer than the expected life of the universe to generate it with a random number generator using all of the compute resources available on the planet. So it's theoretically possible but practically impossible, which makes the OP's statement essentially pedantic.

  while(passes_turing_test(general_ai) == false) {
      general_ai = generate_random_code;
  }

If you can implement passes_turing_test() in a way that is testable without human intervention... that would be a pretty great advance. I'm pretty sure we're not anywhere near there.

The fundamental problem, as I understand it, is that none of us really understand what consciousness is - we all "know it when we see it" - which is a test that is not so amenable to automation. Perhaps if we had enough examples of different kinds of consciousness, we could set up a ML process :)

With regards to what consciousness is, Peter Watts opened an interesting rabbit hole that I have yet to dive into. His discussion about his own real-world inquiries into consciousness is appended at the end of his fiction work "Blindsight" [1]. Search for the heading "Sentience/Intelligence", and read the first couple paragraphs there.

Watts suspects from his readings that sentience makes up the majority of our day-to-day physical and sensorium activities and sapience (consciousness) is a very thin self-aware layer on top of that. If his sources that he's reading are correct, then we can go a long way implementing practical and profitable applications with "just" ML, but with less focus on trying to reach AGI. There are more radical suggestions within the decidedly philosophical side of the cognitive science community that seem to wonder if consciousness isn't simply a very sophisticated, highly complex ML'ish pattern-matching illusion, and we're all reductively representable as deterministic entities somehow.

All this postulating is too far away from the testable engineering/tinkering/science'ing I prefer to inform myself about, so I'm not quite sure what to make of it, other than to observe that ML hasn't made much of a dent in the sentience side, as we're nowhere near a general-purpose robot that cleans floors, does the laundry, and simultaneously avoids squishing the house cat, none of which requires sapience.

[1] http://www.rifters.com/real/Blindsight.htm

"none of us really understand what consciousness is - we all "know it when we see it""

I don't think that's as solid an assumption as your comment makes it sound. We all know magic when we see it, yet it doesn't exist. Many of us recognise when something "feels pain" even if it's a cuddly toy, or recognise "evil presence in the room" when it's gas poisoning or sleep paralysis or biped-shaped-shadows and low frequency sound.

Many of us think we see a level of intelligence where there is none, in perception of God or Evolution "designing" creatures or "coming up with clever solutions", and many refuse to recognise some level of intelligence in humans in cases where they use good reasoning but get the wrong answer, or do something dumb from using the wrong reasoning, and we have ethical/medical arguments about whether people in comas and similar locked-in syndromes are conscious and aware or not, and whether any plants are.

We have continual newspaper headlines along the lines of "scientists recognise {Dolphins, Elephants, $animal} is more intelligent than previously thought" - if we simply "know it when we see it", it should be pretty clear what creatures are conscious and which aren't.

Certainly we can recognise people-like-us as conscious, but to get "enough examples of different kinds of consciousness" we'd have to be able to know it when we see it, and I'm not sure we do.

Haha. Your position, then, is that recognising consciousness is so hard that we don't even know it when we see it?.

That's fair, and in some ways I agree. But I think a certain amount of that is implicit in "I know it when I see it." The phrase implies that there may be significant divergence of opinion.

Some relevant papers:

An Approximation of the Universal Intelligence Measure

https://link.springer.com/chapter/10.1007/978-3-642-44958-1_...

Measuring universal intelligence: Towards an anytime intelligence test

https://www.sciencedirect.com/science/article/pii/S000437021...

I'd probably do it with a writing sample. Say, Macbeth. Just execute randomly generated bytes until they write Macbeth on their own. Of course it could pass on a 100 monkey type fluke, so after it passes then you submit it to a traditional Turing test.

Given sufficient computing resources, this should yield strong AI in just a few minutes.

You could, of course, do machine checks of some subset of "can generate syntactically correct English" -and that would cut out a whole lot more bad programs than 'can you write MacBeth.'

but- even when you pass that test, you will have a nearly infinite number of marcov-chain generators and similar programs to sort through.

My point is that if you want to solve a program by generating random code, you need machine-executable fitness test.

We don't have a machine-executable test for consciousness. The Turing test relies on the intuition of humans, and we have a very finite supply of that.

> even when you pass that test, you will have a nearly infinite number of marcov-chain generators and similar programs to sort through

Not really relevant to your point, but Markov chain generators aren't able to generate syntactically correct English.

(Or, they are able to generate it, but they aren't able to stop themselves from generating syntactically invalid English, and almost all of their output will in fact be invalid.)

I think writing Macbeth isn't a good indicator of anything except being Shakespeare. Asking it a few questions about some general concepts like logic, math, and patterns(relationships in Macbeth maybe?) would probably yield results that are much more likely to pass the test.

I disagree on all counts. The first program that would be generated by such a procedure would very likely be the constant program that happens to always respond with the correct questions to your test. It depends how you randomly generate code, of course, but I doubt the complexity of a "general AI program" would ever be smaller than the complexity of the constant program which by luck returns the correct answer to your test.

Generate the Math tests then? That prevents constants.

Generating macbeth is a prompt most people here couldn’t pass, myself included...

you can still use generating Macbeth as a good indicator, just by providing some inputs and constraints.

Input is a Macbeth outline/play summary (e.g. https://www.cliffsnotes.com/literature/m/macbeth/play-summar... ) Contraints are "in the style of : KingLear and Othello"

and compare the produced text to Macbeth

> Somebody (Geoffrey Hinton, Andrew Ng, I don't remember exactly) made the point that neural networks were in a pretty moribund place, until the confluence of three things came along: more compute power, more data, and algorithmic improvements. Given that, one wonders what other older ideas are waiting to be resurrected by a similar confluence.

Andrew Ng says something like that on his deep learning Coursera course.

Also, Animats mentions perceptrons. A neural network is a generalized (combination of) perceptron(s) on steroids. That ceiling was basically removed again with the advent of the above-mentioned improvements. Similarly, the gradient descent used to optimize a neural network resembles hill climbing, as well.

Reply on news.ycombinator.com