Hacker News Reader

Comment by adrianN | Hacker News Reader

Transistors, while dramatically simpler than neurons, also have fairly complicated physics. It's quite hard to determine the computational power of a microprocessor when you only know roughly how transistors work and have no idea how microprocessors work.

View on HN

True, but transistors are designed by humans (for now), so you have an upper bound on complexity as well as a certain degree of modularity. Whereas biology has no such limitations because evolution doesn't care about the understandability of its designs.

So I tend to err on the side of biology being more complex than not.

But it's absolutely possible that the high-level behaviour is simple while the underlying implementation is complex and chaotic, as in the gas laws.

Probably more to the point is that humans are using transistors in their designs, so we deliberately confine them to their simple modes. It is an oversimplification to talk about transistors being either "on" or "off" because they technically have all sorts of intermediate states, but with a few exceptions, human designs avoid those intermediate states like the plague, because A: they defy our ability to build logic with them and B: depending on those exact behaviors means we can't mass-manufacture chips because the variance of the exact behaviors will be too high.

In principle one could imagine a processor design that works on these intermediate states that somehow vastly exceeds the computational power of a modern system despite using the same base transistors; in practice we have no idea how to build such a thing, and if we did, we wouldn't know how to build a second one of the same thing reliably either.

Biology lacks this restriction. That doesn't mean it's pure and utter chaos, either, there's bounds on that because it still needs systems to at least be metastable. But where humans engineer almost exclusively with stable systems, biology freely uses metastable systems all over the place. And then, even more remarkably, it deals with the question of how to replicate such a strange system in a way that no modern human engineer ever would by making every instance unique, and still somehow functional.

It's a tough act to follow.

Complexity cuts both ways. Complexity, as every programmer knows, is no friend, and can easily create more problems than it solves.

Often the most efficient solution to a problem ends up being a simple one. It would be very surprising if the human brain's Rube Goldberg machine was anywhere near close to a mathematically optimal implementation of intelligence.

It's possible that much of the complexity in a biological neuron is simply working around other complexity introduced by biology, solving problems that we programmers do not even have to think about because we can simply directly use matrix multiplication.

Transistors were invented by humans, but chip layouts themselves involve a lot of automation and at more than one level — Verilog and VHDL both appeared in the 1980s, and there’s tools at both higher and lower levels of abstraction than those two.

Biology hasn't had time to figure out that intelligence is a good idea, up to humans the payoffs have been relatively low. The complexity in human intelligence probably focuses more on doing basic things extremely energy efficiently rather than being effective at thinking.

Humans can attempt to multiply numbers and frequently get the wrong result. That doesn't scream "pushing the limits of intelligence".

I think even dramatically simpler might be an understatement. Neurons are cells, living organisms capable of growth, movement, and some fairly intelligent interaction with their environment. A transistor doesn’t even approach that, it’s more on the level of a single protein within a neuron and not a very complex one.

Don’t forget that transistors are millions of times faster too.

Could indeed be a trade-off between speed vs complexity and efficiency.

Maybe there will be transistor-based human-level AGI soon, but I guess it would require several kilowatts of power compared to the 20 or so watts a human brain requires.

> Maybe there will be transistor-based human-level AGI soon, but I guess it would require several kilowatts of power compared to the 20 or so watts a human brain requires

Given that entire datacenters don't come close to an AGI, let alone a human level one, I fear "several kilowatts of power" is lowballing it by a significant number of orders of magnitude.

More to the point, we don't even know how or what gives rise to a general intelligence, and even defining it is basically a philosophical question. To me, the optimism of some AI enthusiasts (and I don't mean specifically the parent) feels like cavemen contemplating an expedition to the stars shortly after they invented the sling.

I think the chances of us arriving to anything close to an AGI iteratively based on our current capabilities is a pipe dream.

The lowest lowball estimate I’ve seen for the computational capacity of the human brain is 20e15 FLOPS (Kurzweil, 1999), and that would require 2/3 of a megawatt with the current best ranked supercomputer on the Green500 list.

There are already several computers more powerful than this that don’t appear close to a working full brain emulation, and we definitely don’t understand intelligence well enough yet to engineer something like ourselves, so it’s reasonable to be skeptical of estimates saying we’ll be at the kilowatts level “soon” even if it turns out we’re just missing a step which will be obvious in hindsight.

(Unless by “soon” you mean 15 years; I don’t want to bet on anything on that timescale).

> Unless by “soon” you mean 15 years; I don’t want to bet on anything on that timescale

Well, I'm old so by "soon" I mean within my lifetime, e.g. the next 3 to 4 decades.

edit: to clarify a little, when the term AI was first coined by McCarthy in 1956, researchers were confident in cracking AGI within a decade. Then AI-Winter came and people became more cautious. So when I say "soon", I mean it's probably not going to be another 65 years, but also not 5 months or 5 years (unexpected breakthroughs aside).

Faster at what?

But digital chips are designed to use transistors as on/off switches, so the complicated physics don't matter for understanding a microprocessor. (It's different for analog ICs, where the characteristics of the individual transistors do matter)

Yes, but you wouldn't know that if the best knowledge you had about the inner workings of microchips is essentially looking which parts get hot while they perform different tasks.

Wouldn't it suck if brains are just reservoirs used by the internet that can store roughly 3 memes at a time?

holds up a Duracell battery

"You think thermodynamics means using humans as batteries is a dumb idea? Where do you think you learned thermodynamics?"

faint nyan cat music in the background

"May I have a physics textbook?" "No. The real world doesn't run on math."

The paper is measuring I/O behaviour, rather than the complexity of the mechanisms generating that behaviour. Transistors might have quite complex physics, but are designed to have relatively simple I/O behaviour.

Also true - witness the weird circuits generated by evolutionary algorithms on FPGAs etc. where they use chip-specific nonlinear dynamics and capacitative / inductive coupling which only works on the specific chip used to evolve the circuit.

Reply on news.ycombinator.com