... and still so small and slow compared to a human brain.
False, to invent car.
People manage to learn to operate cars in less than 50 hours
The GP is pointing out that training in fine muscle motor skills, self-awareness and ability to project self-awareness to other objects under ones control etc, all took many thousands of years to develop. AI is faster.
However, it's again unfair, as AI only knows what it knows from us, so in that sense any comparison is built on shaky ground.
But for the purposes of comparing a stock human brain as hardware, versus a current high-end GPU specifically in terms of ingesting information and then perform tasks, the GPU beats the human brain "hands-down" in any category.
The only categories it doesn't are simply ones that no one has trained it to yet - so the argument on a pure hardware capability basis stands.
It's just the wrong way to look at the problem. You're not trying to develop the generic system that can learn how to drive a car, you're trying to develop the specific system that can safely drive a car occupied by humans, naturally employing machine learning.
I would argue that we're 95% there, but solving those last 5% is exponentially more expensive, but not commensurately more valuable. There's a "profit ceiling" imposed by the cost of a human driver, which appeats to make solving the problem economically intractable.
Here's a video of Tesla FSD driving through the complicated streets of Los Angeles for an hour straight, with 0 human intervention.
In compare to human's tens of hours?
We have to include our evolutionary process because a lot of our brain is pretrained, especially visual/motor neurons.
We seem to be pretrained to pick up language, for example, and the language(s) we hear after being born fill that space, our brains are plastic for a reason.
On top of typically around 18 years of learning to process and fuse vision, sound, proprioception and other inputs, to navigate the world and reason about it.
2. 16 years of fine-tuning to adapt to the current modern world.
3. 50-100 hours of specific task based fine-tuning for driving, think LORA training.
Power efficiency does not matter for those.
Those are interesting only for the implementation of specialized cognitive functions.
Additionnaly, the connectomes of those chips are 2D and very localized. Human brains are 3D and much less localized. Simulating 3D connectome with those 2D chips slows everything down by a lot.
https://news.mit.edu/2018/study-reveals-how-brain-tracks-obj...
(Though we do need to pay attention to evolution cheating by overfitting relative to what we'd consider a clean design. Some of the complexity may be doing double duty.)
Since we have not succeeded in imitating even the most primitive brains, even though computationally we should have enough juice by now, it would seem that complexity can't be discarded at all, no?
looks at the browser tab with GPT-4 in it
looks back again here
... we didn't?
Seriously, get it to successfully play through a text adventure maze game.
I did exactly that the other day in response to a different objection on a HN thread. Or at least similar enough.
https://cloud.typingmind.com/share/c0a68cb2-5f59-4e83-b383-b...
Now, the goal there wasn't to get it to solve a maze, but rather to see how it can come up with a plan of action and adjust it on the fly. But I see no reason a variant of that wouldn't work with a traditional maze game - provided you remember this is a stateless model without volatile memory, so it needs to be fed its memory with every request.
This is also the reason why its output sounds convincing, but is very often factually wrong.
"imitating even the most primitive brains, even though computationally we should have enough juice by now"
Which is kind of weird to claim today. GPT-4 may be the strongest counterexample to date, but it's far from the only one.
Of course, you need to remember not to confuse the brain with attached peripherals. Just because we can't replicate a perfect worm or fly body, complete with bioelectrical and biomechanical components, doesn't mean we can't do better than their brains in silico.
And if it walks like a duck, and quacks like a duck, ...
GPT-4 is a good example because it's pretty clear that the model isn't merely a stochastic parrot (or, if it is in some sense, then in that sense so are we). But it's not the only game in town. Not all generative transformers deal with language. All seem to be powerful association machines, drawing their capabilities from simple algorithms in absurdly high-dimensional spaces. There are many parallels you can draw to brains here, not the least of which is that the overall architecture is simple enough and scalable, that it's exactly the kind of thing evolution could reach and then get railroaded into building on.
Just like with transformers revolutionising text generation and now things like LoRa and other fine tuning methods are helping us find a better solution to that puzzle, the same will happen for the development of AGIs.
We will do it, one day.
We aren't likely ever going to reduce that to a model as simple as the one used in machine learning, because it probably isn't that simple period.
Neurons are not "just" electrical signalling devices. They are complicated processors and systems in their own right.