Hacker News Reader

> When a programmer successfully emulates consciousness, every line will be deliberate, and they will know exactly how they did it.

I am not so sure. The current success of deep learning seems to be a direct counterexample to this. These techniques are yielding great success by imitating some patterns we observe in the brain, but the resulting inferences more or less defy rigorous explanation. We might fully understand the structure of a neural network but explaining the "why" behind its decisions seems to elude us.

Right. A programmer will never "emulate consciousness", not directly anyway. Rather, a programmer will create a "software brain", which, then, will be subjected to all kinds of experiences, which, in turn, will result in this brain becoming conscious - gradually.

Right. But that "software brain" will be completely deliberate.

> subjected to all kinds of experiences, which, in turn, will result in this brain becoming conscious - gradually.

Again, there is no evidence that "all kinds of experiences" will result in gradual emergence of consciousness from anything we have observed.

To a computer, experience is just data. So we're not talking about computers.

Phrases like "software brain" and "conscious machines" are ways of adding something that is not software to computers. But trust me, if we're talking about software, the programmer will know every line of their code.

Now, if you want to get meta, we can consider programs that write programs. But again, the first program will be written by a human, and we will know exactly what we are doing.

> resulting inferences more or less defy rigorous explanation

We have written programs that take input and produce data models. There is no consciousness here.

Humans take input and produce models too. But I fear you may be confusing learning with consciousness and agency. These not to be found in the code of learning machines, nor in the models they've learned. I am sorry, but it's just not there.

For every abstraction there is an implementation. That is physics. With no implementation, we have metaphysics. Here, "consciousness" is the abstraction. The physics of it is the physical implementation, and is what Roger Penrose and other scientists are working on identifying. The simulation will have a coded implementation. Meaning, whether we generate it or we build something that generates it, when we have "consciousness" we will have the code that backs it. To be able to do this without any understanding of what consciousness is would be analogous to monkeys typing Shakespeare.

But what if intelligence is just a pattern matching ability, and consciousness arises when the entity starts discovering it's place in the patterns?

This reminds me of the very interesting leaps that occur in babies when they are first forming the connections for concepts for things like depth perception, distance, categorization, etc. They are bootstrapping the advanced levels of consciousness that ultimately makes them human in a sense.

>no evidence

Of course there is. The human "wetware" brain does just that.

>know every line of code

But code is there to simply provide some basic mechanisms, not the brain's structure. It is better compared to laws of chemistry or something on that level. It does not need to rewrite itself for the data structures that represent the simulated brain to evolve.

I agree that conscious machines (if such a thing is possible) will probably not be carefully designed by humans.

That said, I think you should be a little more critical of deep learning. So-called neural networks are more meaningfully described as high-dimensional curve fits. When people "train" neural networks, they're fitting the parameters of a model (just like y = a*x + b is a model with a and b as parameters) to some data. The results are impressive, and sometimes downright spooky, but neural networks are closer in spirit to a curve fit in MS Excel than they are to a real brain. There is no deep why. These are just nested compositions of affine transformations and ramp functions.

> as long as it is software, is programmed with a very specific understanding of what is being programmed. Every line of code is intentional. Nothing exists in between the lines.

A reductionist would say that we can explain any phenomenon in the Universe because we know how atoms interact and everything is built out of atoms. When you ask why a tree grows, the answer that it grows because of the interactions between atoms is not useful and approaches the problem on too low level. Similarly a programmer can create artificial neurons and perfectly understand how these neurons operate on a low level, but it doesn't mean the programmer will understand the high level emergent phenomenons that can result from these low level interactions.

> AI is perfectly guided. Every program is completely understood (insert incompetent programmer joke here). Every symbol is typed with purpose on purpose.

This is not at all how modern AI systems work. Modern AI systems work by feeding lots of data into a system that uses randomized heuristics to come up with an answer that has high confidence. You change the training data a little bit and the output can be entirely different, for reasons that are far from clear.

I think every line of code is not intentional. People copy code, they make mistakes, there is emergent behavior. If you perhaps worked on a large system, like I have, databases, the database optimizers do things that you didn't necessarily intend. the plan choices often get pushed in certain directions that were not necessarily what you meant. I worked on a query optimizer, and I tried to model the cost of different kinds of scans and filters, say on ints, vs strings, vs floats. It turned out strings were getting picked a lot of the time, and my choices (my modeling, really), led us to priotize using string based indexes more. Certain bugs and limitations in cardinality estimation lead to accidental but fortuitiously good query plan choices that were sometimes lost (not chosen) when we fixed certain bugs, exposing other bugs.

I definitely believe things happen in large programs that no one really intended.