Hacker News Reader

Comment by Kapura | Hacker News Reader

This gets at the fundamental issue I have with AI: We're trying to get machines to think, but we only have the barest understanding of how we, as humans, think. The concept of a "neuron" that can activate other neurons comes straight from real neurology, but given how complex the human brain is, it's no surprise that we can only create something that is fundamentally "lesser."

However, I think that real neurology and machine-learning can be mutually reinforcing fields: structures discovered in the one could be applied to the other, and vice versa. But thinking we can create "AGI" without first increasing our understanding of "wet" neural nets is the height of hubris.

View on HN

All the efforts to get computers to play chess mostly like humans do were made to look insufficient by a large enough neural network that took zero knowledge of human position evaluation, which just played against itself an outrageous number of games.

The plane is in many ways lesser than birds, and we barely understood aerodynamics when the wright brothers gave us a working plane: While the bird is efficient, we had a whole lot more thrust. Our planes are far better than they were back then, but not because we understand bird better, but because we focused on efficiency of the simpler designs we could build. The Formula 1 car doesn't come from deep understanding of the efficient, graceful running movements of the cheetah.

Our results in medicine have been far and ahead of our understanding of biochemistry, DNA, and in general, how the human body works. The discovery of penicilin didn't require a lot of understanding: It required luck. Ozempic and Viagra didn't come from deep understanding of the human body: Resarchers were looking for one thing, and ended up with something useful that was straight out unintended.

We might be able to build AGI with what we know of brains, we might not. But either way, it's not because we need to understand the human mind better: Do we have enough compute for a very crude systems that we know how to build to overcome our relative lack of understanding?

So if you ask me, what is the height of hubris is to think that working engineering solutions have to come from deep understanding of how nature solves the problem. The only reasonable bet, given the massive improvements in AI in the last decade, is to assume that the error bars of any prediction here are just huge. Will we get stuck, the way self-driving cars seem to have gotten stuck for a bit? Possibly. Will be go from a stochastic parrot into something better than humans, the same way that Go AIs went from pretty weak to better than any human in the blink of an eye? I'd not discount it either. I expect to be surprised either way.

> height of hubris is to think that working engineering solutions have to come from deep understanding of how nature solves the problem.

It is basically understood how "nature solves the problem". It does that by evolution. And what is machine learning and neural networks but artificial evolution?

Nothing of the sort -- it's best characterized as learning, like it says in the name.

Something like a GAN is closer to what we'd consider evolution.

You only save the changes that improve the loss values no? Those are the survivors whose progeny go on to the next iteration.

> It is basically understood how "nature solves the problem". It does that by evolution.

Also known as throwing shit at a wall to see what sticks. Evolution is dumb, but it brute-forces its way down the optimization gradients by sheer fucking scale of having turned the entire surface of this planet into tiny, imperfectly self-replicating machines.

In AI research there are different branches with different goals and motivations. You have applied research, where the goal is to engineer systems that are better at soving certain needs. You also have scientific endeavours, usually interdisciplinary, typically crossed with biology, psychology and philosophy, where AI models are created and used as models of understanding human or animal intelligence.

The former is where as you describe where better understanding of biology might help but is not a prerequisite for progress, but in the latter it is not just needed but the goal.

Now I know this is a bit of a caricature as both of these disciplines are in practice poorly deliniated and often intermixed. It's easy to find hubristic examples to mock on both sides, but there's value and brilliance to be found in each respectively as well.

> All the efforts to get computers to play chess

You say Go later so I think you know this and it was just a thinko, but chess fell easily to pre-neural techniques that are fully logic based and understandable. It was Go that required a big neural network trained through self play.

I think what they meant was that even logic-based superhuman chess was overtaken by AlphaZero.

AGI and human intelligence don't necessarily have to work the same way. While many of us assume that they must be very similar, I don't think there's any basis for that assumption.

That's correct that they don't need to work the same way. I would say most people that have earnestly thought about it assume they won't.

But given the exponentially greater chemical, biological, and neurological complexity of human brains, the millions of years of evolutionary "pre-training" that they get imbued with, and the years of constant multimodal sensory input required for their culmination in what's called human intelligence, it takes an extremely bold assumption to believe that equivalent ends can be met with a stream of fmadds driving through in an array of transistors.

It's hard to overstate how big a gulf there is in both the hardware and software between living systems and what we currently see in our silicon projects. Neural network learning in general and LLM's in particular are like crude paper airplanes compared to the flight capabilities of a dextrous bird. You can kinda squint and see that they both move some distance through the air, and even marvel at it for a bit, but there's a long long long long way to go.

And yet no bird ever broke the sound barrier. Humans make things that exceed anything in nature. Thought will only be another example.

Exceed in some regards but not others right? Like, let’s not imagine the plane is better at being a bird than the bird. It’s an evocative set of associations though. Makes me think we’ll end up creating the Concorde of intelligence but be no closer to understanding how our own works…

It'll explain it to us, at least to the best of our capability, with lots of hand-holding.

Or it won't and you'll wonder why your molecular constituents are being siphoned off to build things it finds more useful.

If it sticks my brain in a simulation to keep me occupied, I'm totally ok with this.

Reminder: a tamagotchi Workaccount2 is not you. A parallel instance of you, no matter how perfect, cannot add to your qualitative experiences or make you less dead.

I agree that creating something akin to an organic being exceeds capabilities at present. But to be fair we never tried to build a bird, instead we tried to build a machine to fly and we excelled.

However I concur that we may end up creating an AI which thinks nothing like us, has no sense of morality, and yet far exceeds us in intelligence. And that would be somewhat unsettling.

Without realizing it you're voicing the concerns of the AI doomers.

We may build something that far exceeds us in capability, but without us understanding it, or it understanding us. This is the alignment issue.

I do wonder if such a thing can actually exist in the realm of intelligence like in the realm of, say, flight. It’s not clear that’s the case.

> Exceed in some regards but not others right?

Yeah, nature does it bottom-up through evolution of self-assembling, self-replicating machines. A lot of "design constraints" that go into living things has to do with... keeping them alive. Compare an electrical wire or silicon trace with a biological nerve or neuron - the former are just simple traces of inert metal, the latter are complex nanomachines mostly dedicated to self-maintenance, and incidentally conducting electricity.

Point being, we can't compete with nature across every dimension simultaneously, but we don't need to, and we shouldn't, because we don't need most of the features of natural solution, not at this point. We don't need Concordes hatching from eggs - we have factories that build them.

This basis is our limited data set: we have only 1 example of human level intelligence, so it's natural to assume there's something unique, or at least significant about the way it works.

> We're trying to get machines to think, but we only have the barest understanding of how we, as humans, think... But thinking we can create "AGI" without first increasing our understanding of "wet" neural nets is the height of hubris.

That depends on the landscape of the solution space. If the techniques that work much better than anything else happen to be the same techniques used within our own brains, then we might find them, through aggressive experimentation and exploration, before we even realize that it's also how our own brains work too.

We do in fact know that whatever makes the brains tick, it must be something 1) simple enough for the extremely dumb, brute-force optimizer that is evolution to stumble on it, 2) incrementally improvable by the same mechanism, and 3) conferring some kind of advantage at every step. Those are requirements for some feature to evolve.

3 is not required. Multiple steps with no negatives is sufficient.

> thinking we can create "AGI" without first increasing our understanding of "wet" neural nets is the height of hubris

Already ~40 years ago I read in a popular science book that scientists had tried to apply genetic algorithms to finding a better nozzle for a given purpose (jet engine? I don't remember that). That was then done by prefabricating a number of ring-shaped metal parts the inside of which had different widths and different slopes; a stack of these could form a wide variety of shapes for the conduit. They started with an initial configuration, did a measurement, then threw the dice to determine what part should be changed; if the new measurement was better (according to their metric) than the preceding one, they kept it, otherwise reversed it. They did find a nozzle shape that was not only significantly better than any known configuration; it was also significantly weirder, seemingly completely ad-hoc.

There's no computers in this experiment, no intelligent design; people were only necessitated to procure the initial parts and setup, do the dice-throwing and so on; the design process proper was a very basic, randomized, mechanical procedure, no thinking required, no understanding necessary.

I see no reason why this is a problem. We built analogs of many things before we fully understood how they work. Most of human progress has worked this way through an empirical approach. Makes sense since the real world is the ultimate sandbox and the final judge of an idea’s applicability.

Hard disagre. Imho humans suffer under a delusion that their thinking is like a computer whereas it is actually much more like an LLM. So if you ask humans[1] to figure our how they think, the answer will be a) long time coming and b) wrong.

[1] possibly bhudist monks excepted.

Reply on news.ycombinator.com