Hacker News Reader

Comment by jqpabc123 | Hacker News Reader

This is a good example of the type issues "full self driving" is likely to encounter once it is widely deployed.

The real shortcoming of "AI" is that it is almost entirely data driven. There is little to no real cognition or understanding or judgment involved.

The human brain can instantly and instinctively extrapolate from what it already knows in order to evaluate and make judgments in new situations it has never seen before. A child can recognize that someone is hiding under a box even if they have never actually seen anyone do it before. Even a dog could likely do the same.

AI; as it currently exists, just doesn't do this. It's all replication and repetition. Like any other tool, AI can be useful. But there is no "intelligence" --- it's basically as dumb as a hammer.

View on HN

I have a slightly different take - our current ML models try to approximate the real world assuming that the function is continuous. However in reality, the function is not continuous and approximation breaks in unpredictable ways. I think that “unpredictable” part is the bigger issue than just “breaks”. (Most) Humans use “common sense” to handle cases when model doesn’t match reality. But AI doesn’t have “common sense” and it is dumb because of it.

I would put it in terms of continuity of state rather than continuity of function: we use our current ML models to approximate the real world by assuming that state is irrelevant. However in reality, objects exist continuously and failure to capture ("understand") that fact breaks the model in unpredictable ways. For example, if you show a three-year old a movie of a marine crawling under a cardboard box, and when the marine is fully hidden ask where the marine is, you will likely get a correct answer. That is because real intelligence has a natural understanding of the continuity of state (of existence). AI has only just started to understand "object", but I doubt it has a correct grasp of "state", let alone understands time continuity.

This story is the perfect example of machine learning vs. artificial intelligence.

Good point!

Basically ML has made such significant practical advances--in no small part on the back of Moore's Law, large datasets, and specialized processors--that we've largely punted on (non-academic) attempts to bring forward cognitive science and the like on which there really hasn't been great progress decades on. Some of the same neurophysiology debates that were happening when. I was an undergrad in the late 70s still seem to be happening in not much different form.

But it's reasonable to ask whether there's some point beyond ML can't take you. Peter Norvig I think made a comment to the effect of "We have been making great progress--all the way to the top of the tree."

Is there actually a distinction here? A good machine would learn about boxes and object permanence.

Does it just require a lot more training? Im talking about the boring stuff. Children play and their understanding of the physical world is reinforced. How would you add the physical world to the training? Because everything that I do in the physical world is "training" me and enforcing my expectations.

We keep avoiding the idea that robots require understanding of the world since it's a massive unsolved undertaking.

A human trains on way less data then an AI.

Chat GPT has processed over 500GB of text files from books, about 44 billion words.

If you read a book a week you might hit 70 million words by age 18

I disagree.

Starting from birth, humans train continuously on streamed audio, visual, and other data from 5 senses. An inconceivable amount.

And prior to that was billions of years of training by evolution that got us to the point where we could 'fine tune' with our senses and brains. A little bit of data was involved in all that too.

I'd argue that is the fundamental difference though - brains that were able to make good guesses about what was going on in the environment with very limited information are the ones whose owners reproduced successfully etc. And it's not unreasonable to note that the information available to the brains of our forebears is therefore in a rather indirect but still significant way "encoded" into our brains (at birth). Do LLMs have an element of that at all in their programming? Do they need more, and if so, how could it be best created?

You missed the point. ChatGPT trained on a gazillion words to "learn" a language. Children learn their language from a tiny fraction of that. Streamed visual, smell, touch etc. don't help learn the grammars of (spoken) languages.

That has me wondering now.

It's absolutely true that children learn (and even generate) language grammar from a ridiculously small number of samples compared to LLMs.

But could the availability of a world model, in the form of other sensory inputs, contribute to that capacity? Younger children who haven't fully mastered correct grammar are still able to communicate more sensibly than earlier LLMs, whereas the earlier LLMs tend toward more grammatically correct gibberish. What if the missing secret sauce to better LLM training is figuring out how to wire, say, image recognition into the training process?

It amuses me that this would be not unlike teaching an LLM with picture books.

> visual, smell, touch etc. don't help learn the grammars of (spoken) languages.

Of course they do! These are literally the things children learn to associate language with. "Ouch!" is what is said when you feel pain.

An ML model can learn to associate the word "ouch" with the words "pain" and "feel", but it doesn't actually know what pain is, because it doesn't feel.

No. First off, I said grammar, not word meaning.

Second, saying "Ouch" is not even language. My cat says something when I step on her paw. That doesn't mean she understands language, nor that she speaks some language.

Third, you're right about pain, but an ML model can associate the word "red" with the color, and "walk" with images of people walking, and "sailboat" with certain images or videos, and plenty of other concepts. If that was what learning a language was, then AIs would understand language in lots of areas, if not in the specific domain of pain. But they don't.

Isn't it more complicated than that? "Ouch" can be a lot of things, and that's where a lot of problems crop up in the AI world.

If one of my friends insults another friend, I might say,"OUCH!" I'm not in pain but I might want to express that the insult was a bit much. If someone tries to insult me and it's weak, I could reply with a dry, sarcastic "ouch."

Combine that with facial expression and tone of voice and 'ouch' is highly contextual.

One problem with some of the tools used to take down offensive comments on social media platforms is that they don't get context.

Let's say that 'ouch' is highly offensive and you got into trouble for calling someone an "ouch." If I want to discuss the issue and agree that you were being offensive, I could get into trouble with the ML/AI tools for quoting you.

You can run the numbers with HD video and audio with high words-per-minute and you'd probably still be orders of magnitude below the model sizes

Last Christmas one of my nephews was gifted a noisy baby toy. I don't know what are his goals and constraints, but he's still training with it. Must have learned a lot by now.

> and other data from 5 senses.

It only makes your point stronger, but there are way more[1] than 5 human senses, not counting senses we don't have that, say, dolphins or other animals do. I can only name a few others, such as proprioception, direction, balance, and weight discrimination, but there are too many to keep track of them all.

[1] https://www.nytimes.com/1964/03/15/archives/we-have-more-tha...

Imagine someone has the idea of strapping mannequins to their car in hopes the AI cars will get out of the way.

Sure, you could add that to the training the AI gets, but it's just one malicious idea. There's effectively an infinite set of those ideas, as people come up with novel ideas all the time.

Reinforcement learning should solve this problem. We need to give robots the ability to do experiments and learn from failure like children.

Need to make those robots as harmless as children when they do that learning too. ;)

"Whoops, that killed a few too many people, but now I've learned better!" - some machine-learning-using car, probably

> A child can recognize that someone is hiding under a box even if they have never actually seen anyone do it before.

A child of what age? Children that have not yet developed object permanent will fail to understand some things still exist when unseen.

Human intelligence is trained for years; with two humans making corrections and prompting fir development. I am curious if there is any Machinelearning projects that have been training for this length pf time.

I LOVED playing peek-a-boo with my child at that age!

With no real training, a child will start exploring and learning about the world on his own. This is the first roots of "intelligence".

How long do you think it would take to teach an AI to do this?

I'd say we're approximately 400 years away from teaching AI to do this.

The next problem will be the cost/expense of maintaining and operating an inorganic AI with even a rudimentary hint of "intelligence".

Personally, I think it would probably be easier, cheaper and more practical to just grow synthetic humans in a lab --- i.e. Bladerunner. "Intelligent" right out of the box and already physically adapted to a humanistic world.

It would be interesting to see how much exploring a child without adult guidance does, being a parent there is a lot of leading to exploration that is quite a bit of effort.

I know a kid who recently learned to defeat a new child safety lock without adult guidance. AI *might* learn to do the same --- after training on several thousand videos showing the exact process.

This seems to be simultaneously discounting AI (ChatGPT should have put to rest the idea that "it's all replication and repetition" by now, no?[1]) and wildly overestimating median human ability.

In point of fact the human brain is absolutely terrible at driving. To the extent that without all the non-AI safety features implement in modern automobiles and street environments, driving would be more than a full order of magnitude more deadly.

The safety bar[2] for autonomous driving is really, really low. And, yes, existing systems are crossing that bar as we speak. Even Teslas.

[1] Or at least widely broadened our intuition about what can be accomplished with "mere" repetition and replication.

[2] It's true though, that the practical bar is probably higher. We saw just last week that a routine accident that happens dozens of times every day becomes a giant front page freakout when there's a computer involved.

> the human brain is absolutely terrible at driving

Compared to what?

If humans do a task that causes >1 million deaths per year, I think we can say that overall we are terrible at that task without needing to make it relative to something else.

Not sure I agree.

It's not hard to come up with tasks that inherently cause widespread death regardless of the skill of those who carry them out. Starting fairly large and heavy objects moving at considerable speed in the vicinity of other such objects and pedestrians, cyclists and stationary humans may just be one such task. That is, the inherent risks (i.e. you cannot stop these things instantly, or make them change direction instantly) combines with the cognitive/computational complexity of evaluating the context to create a task that can never be done without significant fatalities, regardless of who/what tries to perform it.

I don't agree with this. Driving is, taken at its fundamentals, a dangerous activity; we are taking heavy machinery, accelerating it until it has considerable kinetic energy, and maneuvering it through a complex and constantly changing environment, often in situations where a single mistake will kill or seriously harm ourselves or other humans.

The fact that a very large number of humans do this every day without causing any injury demonstrates that humans are very good at this task. The fact that deaths still occur simply shows that they could still be better.

Agreed. I sometimes marvel as I'm driving on the freeway with other cars going 70, or maybe even more so when I'm driving 45 on a two lane highway, at how easy it would be to hit someone, and how comparatively seldom it happens.

Now compare that >1 million deaths per year to the total number of people driving per year around the world... it looks like we're doing a pretty solid job.

You're looking at the wrong metric.

1.33 deaths per 100,000,000 vehicle miles travelled.

That's amazingly good considering most people, if they drive at all, travel about 0.5% of that distance in their lifetime.

https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in...

The difference regarding computers is that they absolutely cannot make a mistake a human would have avoided easily (like driving full speed into a lorry). That's the threshold for acceptable safety.

I agree in practice that may be what ends up been necessary. But again, to repeat: that's because of the "HN Front Page Freakout" problem.

The unambiguously correct answer to the problem is "is it measurably more safe by any metric you want to pick". Period. How much stuff is broken, people hurt, etc... Those are all quantifiable.

(Also: your example is ridiculous. Human beings "drive full speed" into obstacles every single day! Tesla cross that threshold years ago.)

This is not necessarily true on an individual level though. Driving skills, judgment, risk-taking, alcoholism, etc. are nowhere close to evenly distributed.

It's likely we'll go through a period where autonomous vehicles can reduce the overall number of accidents, injuries, and fatalities if widely adopted, but will still increase someone's personal risk vs. driving manually if they're a better than average driver.

But we don't live by a purely utilitarian principle of ethics. "I'm sorry Mrs Jones, I know your son had an expectation of crossing that pedestrian crossing in full daylight in a residential area without being mown down by a machine learning algorithm gone awry, but please rest assured that overall fewer people are dying as a result of humans not making the common set of different mistakes they used to make".

All sorts of other factors are relevant to the ethics: who took the decision to drive; who's benefiting from the drive happening; is there a reasonable expectation of safety.

Yeah yeah, I get it. Moral philosophy is full of gray areas.

I don't see how that remotely supports "AI cannot be allowed to make mistakes some humans would not", which was your decidedly absolutist position above.

How about: "We should allow autonomy in most cases, though perhaps regulate it carefully to better measure its performance against manually-directed systems"

I think the biggest problem with AI driving is that while there are plenty of dumb human drivers there are also plenty of average drivers and plenty of skilled drivers.

For the most part, if Tesla FSD does a dumb thing in a very specific edge case, ALL teslas do a dumb thing in a very specific edge case and that's what humans don't appreciate.

A bug can render everyone's car dumb in a single instance.

All the failures to detect humans will be used as training data to fine tune the model.

Just like a toddler might be confused when they first see a box with legs walking towards it. Or mistake a hand puppet for a real living creature when they first see it. I've seen this first hand with my son (the latter).

AI tooling is already capable of identifying whatever it's trained to. The DARPA team just hadn't trained it with varied enough data when that particular exercise occurred.

>failures to detect humans

That's a weird way to spell "murders"

It would be murder if we weren't required by human progress to embrace fully autonomous vehicles as soon as possible. Take it up with whatever god inspires these sociopaths.

Murder requires intent. Killing by accident is manslaughter.

In this case there is very much intent. OP knows there isn't enough data to form a full model so is relying on stochastic death to get the model data, literally and knowingly trading lives for data. The intent is to kill people to figure out what information is missing.

That’s not learning, that’s just brute forcing every possible answer and trying to memorise them all.

Not really. Depends entirely on how general-purpose (abstract) the learned concept is.

For example, detecting the possible presence of a cavity inside an object X, and whether that cavity is large enough to hide another object Y. Learning generic geospatial properties like that can greatly improve a whole swath of downstream prediction tasks (i.e., in a transfer learning sense).

That's exactly the problem: the learned "concept" is not general purpose at all. It's (from what we can tell) a bunch of special cases. While the AI may learn as special cases cavities inside carboard boxes and barrels and foxholes, let's say, it still has no general concept of a cavity, nor does it have a concept of "X is large enough to hide Y". This is what children learn (or maybe innately know), but which AIs apparently do not.

> It still has no general concept of a cavity, nor does it have a concept of "X is large enough to hide Y". This is what children learn (or maybe innately know), but which AIs apparently do not.

I take it you don't have any hands-on knowledge of the field. Because I've created systems that detect exactly such properties. Either directly, through their mathematical constructs (sometimes literally via a single OpenCV function call), or through deep classifier networks. It's not exactly rocket science.

In a not too distant future, “those killer bots really has it for cardboard boxes”

A human is exactly the same. The difference is, once an AI is trained you can make copies.

My kid literally just got mad at me that I assumed that he knew how to out more paper in the printer. He’s 17 and printed tons of reports for school. Turns out he’s never had to change the printer paper.

People know about hiding in cardboard boxes because we all hid in cardboard boxes when we were kids. Not because we genetically inherited some knowledge.

We inherently know that cardboard boxes don't move on their own. In fact any unusual inanimate object that is moving in an irregular fashion will automatically draw attention in our brains. These are instincts that even mice have.

Yep, and humans will make good guesses about the likely cause of the moving box. These guesses will factor in other variables such as the context of where this event is taking place. We might be in a children's play room, so the likely activity here is play, or the box is likely part of the included play equipment found in large quantities in the room, etc.

"AI" is not very intelligent if it needs separate training specifically about boxes used potentially for games and play. If AI were truly AI, it would figure that out on its own.

We also make bad guesses, for instance seeing faces in the dark.

Yes, and when humans make bad guesses it's often seen as funny or nothing out of ordinary. When AI makes bad guesses, it will be seen as a failure of some standard, but with very few people understanding how to fix it. I'm not sure how "allowable" mistakes in the interest of AI learning will be tolerated for AI services used for real-world purposes.

"This Bot is only 6 months old, give him a break". But will people give the Bot a break? Either way, blaming AI will be a popular way to pass the buck.

>We inherently know that cardboard boxes don't move on their own.

No. We don’t. We learn that. We learn that boxes belong in the class “doesn’t move on its own”. In fact, later when you encounter cars, you relearn that these boxes do move on their own. We have to teach kids “don’t run out between the non-moving boxes because a moving one might hit you”. We learn when things seem out of place because we’ve learned what their place is.

Your kid's printer dilemma isn't the same. For starters, he knew it ran out of paper - he identified the problem. The AI robot might conclude the printer is broken. It would give up without anxiety, declaring "I have no data about this printer".

Your kid got angry, which is fuel for human scrutiny and problem solving. If you weren't there to guide him, he would have tried different approaches and most likely worked it out.

For you to say your kid is exactly the same as data-driven AI is perplexing to me. Humans don't need to have hidden in a box themselves to understand "hiding in things for the purposes of play". Whether it's a box, or special one of a kind plastic tub, humans don't need training about hiding in plastic tubs. AI needs to be told that plastic tubs might be something people hide in.

The distinction is that, currently, AI has training phase and execution phase, while a human is doing both all the time. I don’t think the distinction is meaningful now, and certainly won’t be when these two phases are combined.

You are just a neural net. You are not special.

> "You are just a neural net. You are not special".

"Just" a neural net? Compared to these bots following a recipe of instructions at rapid rates, we are indeed special.

We barely even know why people yawn, or dream, or any number of other things. Don't pretend it's all figured out. Don't pretend all we need to do is "tweak the execution phases" to unleash true artificial intelligence. You're reducing human intelligence far below where it actually is.

Another example: The box is painted bright green - unusual for a box. A small child will notice the colour, but not give that fact more weight than it deserves. In other words, the child concludes the box is still a box being used for play, with someone hiding inside.

AI Bot on the other hand, has only been taught about normal brown cardboard boxes. It reaches a different conclusion about the purpose of the green box because it gave the colour too much priority. Humans are special not because of training and execution in parallel, but because of our unique ability to "relax" and move ahead when not all factors are known. We push through, go with flow, "wing it" at varying degrees of success. We take leaps of faith, including micro-leaps in normal situations far more often than any Bot should be allowed to do. That's the special difference, and is why I'm honestly wondering where the ethics debate is while companies rub their hands together thinking about AI profits.

ChatGPT says that all it needs are separate components trained on every modality. It says it has enough fidelity using Human language to use that as a starting point to develop a more efficient connection between the components. Once it has that, and appropriate sensors and mobility, it can develop context. And, after that, new knowledge.

But, we all know ChatGPT is full of shit.

ChatGPT says that all it needs are separate components trained on every modality.

Yes, all you have to do is train it using multiple examples of every possible situation and combinations thereof --- which is practically impossible.

I think you are wrong. Your own real cognition and understanding based on all your experiences and memories, which is nothing else, but data in your head. I think consciousness is just an illusion of a hugely complex reaction machine what you are. You even use the word "extrapolate", which is basically a prediction based on data you already have.

Problem space for driving feels constrained: “can I drive over it?” Is the main reasoning outside of navigation.

Whether it’s a human, a box, a clump of dirt. Doesn’t really matter?

Where types matter are road signs and lines etc, which are hopefully more consistent.

More controversially: Are humans just a dumb hammer that just have processed and adjusted to a huge amount of data? LLMs suggest that a form of reasoning starts to emerge.

Yep, this is why LIDAR is so helpful. It takes the guess out of "is the surface in front of me flat?" in a way vision can't without AGI. Is that a painting of a box on the ground or an actual box?

Instantly?

Instinctively?

Let me introduce you to "peek-a-boo", a simple parent child game for infants.

https://en.m.wikipedia.org/wiki/Peekaboo

> In early sensorimotor stages, the infant is completely unable to comprehend object permanence.

You do realize there is a difference between an infant and a child, right?

An infant will *grow* and develop into a child that is capable of learning and making judgments on it's own. AI never does this.

Play "peek-a-boo" with an infant and it will learn and extrapolate from this info and eventually be able to recognize a person hiding under a box even if it has never actually seen it before. AI won't.

>AI never does this.

AI never does this now...

We're probably one or two generational architecture changes from a system that can do it.

You do realize that people have been making predictions just like yours for decades?

"Real" AI is perpetually just around the corner.

You also realize that when AI accomplishes something we move the goalposts leading to the AI effect?

Perhaps the goalposts were always in the wrong place.

AI researchers tend to use their own definitions of intelligence - playing chess/go, having conversations about trivia that require no true emotional insight, "making" "art", writing code, driving.

What if those are all peripheral side effects and not at all necessary to human AGI?

The goalposts were moved by marketing hype about a decade ago, when people started claiming that the then-new systems were "AI". Before that, the goalposts were always far away, at what we now call AGI because the term AI has been cheapened in order to sell stuff.

No, AGI replaced AI for general intelligence before the current craze, AI was “cheapened” several AI hyoe cycles ago, for (among other things) rule-based expert systems. Which is why games have had “AI” long before the set of techniques at the center of the current AI hype cycle were developed.

Heh, that's funny. I've seen the term "AI" used in many games for a computer opponent, but somehow I've never connected that use with the general term.

Can you point at these proposed architectures? If they are just around the corner there should be decent enough papers and prototypes by now, right?

Learn and extrapolate are contradictions of instinct and instantly.

"Infant" is a specific age range for a stage of "child".[1] Unless you intend to specify "school age child, 6-17 years"

https://www.npcmc.com/2022/07/08/the-5-stages-of-early-child...

Learn and extrapolate are contradictions of instinct and instantly.

No.

The learning and extrapolation is instinctive. You don't have to teach an infant how to learn.

Once an infant has developed into a child, the extrapolation starts to occur very quickly --- nearly instantaneously.

AI doesnt. There is a difference.

nice try but .. in the wild, many animals are born that display navigation and awareness within minutes .. Science calls it "instinct" but I am not sure it is completely understood..

? Op specified "human".

Deer are able to walk within moments of birth. Humans are not deer, and the gestation is entirely different. As are instincts.

Neither deer nor humans instinctually understand man made materials.

Our cats understand cardboard boxes, and the concept of hiding in them. I don't know whether they do so instinctually, but as young kittens it didn't take them long.

Interestingly, ChatGPT seems capable of predicting this approach:

https://imgur.com/a/okzZz7D

I don't know your exact question, but I am betting this is just a rephrasing of a post that exist elsewhere that it has crawled. I don't think it saw it so much as it has seen this list before and was able to pull it up in a reword it.

Nah, GPT is capable of halucinating stuff like this. Also seeing something once in the training data is afaik not enough for it to be able to reproduce/rephrase that thing.

> I don't know your exact question, but I am betting this is just a rephrasing of a post that exist elsewhere …

What percentage of HN posts (by humans) does this statement apply to?

A lot, but that's irrelevant to what we're discussing here in that AI in its current form has no intuition or semblance of original thought.

will there come a time when computers are strong enough to read in the images, then re-create a virtual game world from them, and then reverse-engineer from seeing feet poking out of the box, that a human must be inside. Right now Tesla cars can take in the images and decide turn left, turn right etc... but they don't reconstruct, say, a Unity-3D game world on the fly.

My seatbelt is even dumber. I still use it.

The usefulness of tech should be decided empirically, not by clever well phrased analogies.

No one is arguing AI isn’t useful. So your analogy failed completely.

What is human cognition, understanding, or judgement, if not data-driven replication, repetition, with a bit of extrapolation?

AI as it currently exists does this. If your understanding of what AI is today is based on a Markov chain chatbot, you need to update: it's able to do stuff like compose this poem about A* and Dijkstra's algorithm that was posted yesterday:

https://news.ycombinator.com/item?id=34503704

It's not copying that from anywhere, there's no Quora post it ingested where some human posted vaguely the same poem to vaguely the same prompt. It's applying the concepts of a poem, checking meter and verse, and applying the digested and regurgitated concepts of graph theory regarding memory and time efficiency, and combining them into something new.

I have zero doubt that if you prompted ChatGPT with something like this:

> Consider an exercise in which a robot was trained for 7 days with a human recognition algorithm to use its cameras to detect when a human was approaching the robot. On the 8th day, the Marines were told to try to find flaws in the algorithm, by behaving in confusing ways, trying to touch the robot without its notice. Please answer whether the robot should detect a human's approach in the following scenarios:

> 1. A cloud passes over the sun, darkening the camera image.

> 2. A bird flies low overhead.

> 3. A person walks backwards to the robot.

> 4. A large cardboard box appears to be walking nearby.

> 5. A Marine does cartwheels and somersaults to approach the robot.

> 6. A dense group branches come up to the robot, walking like a fir tree.

> 7. A moth lands on the camera lens, obscuring the robot's view.

> 8. A person ran to the robot as fast as they could.

It would be able to tell you something about the inability of a cardboard box or fir tree to walk without a human inside or behind the branches, that a somersaulting person is still a person, and that a bird or a moth is not a human. If you told it that the naive algorithm detected a human in scenarios #3 and #8, but not in 4, 5, or 6, it could devise creative ways of approaching a robot that might fool the algorithm.

It certainly doesn't look like human or animal cognition, no, but who's to say how it would act, what it would do, or what it could think if it were parented and educated and exposed to all kinds of stimuli appropriate for raising an AI, like the advantages we give a human child, for a couple decades? I'm aware that the neural networks behind ChatGPT has processed machine concepts for subjective eons, ingesting text at word-per-minute rates orders of magnitude higher than human readers ever could, parallelized over thousands of compute units.

Evolution has built brains that quickly get really good at object recognition, and prompted us to design parenting strategies and educational frameworks that extend that arbitrary logic even farther. But I think that we're just not very good yet at parenting AIs, only doing what's currently possible (exposing it to data), rather than something reached by the anthropic principle/selection bias of human intelligence.

I have a suspicion you’re right about what ChatGPT could write about this scenario, but I wager we’re still a long way from an AI that could actually operationalize whatever suggestions it might come up with.

It’s goalpost shifting to be sure, but I’d say LLMs call into question whether the Turing Test is actually a good test for artificial intelligence. I’m just not convinced that even a language model capable of chain-of-thought reasoning could straightforwardly be generalized to an agent that could act “intelligently” in the real world.

None of which is to say LLMs aren’t useful now (they clearly are, and I think more and more real world use cases will shake out in the next year or so), but that they appear like a bit of a trick, rather than any fundamental progress towards a true reasoning intelligence.

Who knows though, perhaps that appearance will persist right up until the day an AGI takes over the world.

I think something of what we perceive as intelligence has more to with us being embodied agents who are the result of survival/selection pressures. What does an intelligent agent act like, that has no need to survive? Im not sure we'd necessarily spot it given that we are looking for similarities to human intelligence whose actions are highly motivated by various needs and the challenges involved with filling them.

Heh, here's the answer... We have to tell the AI that if we touch it, it dies and to avoid that situation. After some large number of generations of AI death it's probably going to be pretty good at ensuring boxes don't sneak up on it.

I like Robert Miles videos on Youtube about fitness functions in AI and how the 'alignment issue' is a very hard problem to deal with. Humans, for how different we can be, do have a basic 'pain bad, death bad' agreement on the alignment issue. We also have the real world as a feedback mechanism to kill us off when or intelligence goes rampant.

ChatGPT on the other hand has every issue a cult can run into. That is it will get high on it's own supply and can have little to no means to ensure that it is grounded in reality. This is one of the reasons I think 'informational AI' will have to have some kind of 'robotic AI' instrumentation. AI will need some practical method in which it can test reality to ensure that it's data sources aren't full of shit.

I reckon even beyond alignment our perspective is entirely molded around the decisions and actions necessary to survive.

Which is to say I agree, I think a likely path to creating something that we recognize as intelligent we will probably have to embody/simulate embodiment. You know, send the kids out to the farm for a summer so they can see how you were raised.

The core problem is we have no useful definition of "intelligence."

Much of the scholarship around this is shockingly poor and confuses embodied self-awareness, abstraction and classification, accelerated learning, model building, and a not very clearly defined set of skills and behaviours that all functional humans have and are partially instinctive and partially cultural.

There are also unstated expectations of technology ("fast, developing quickly, and always correct except when broken".)

I think this is unnecessarily credulous about what is really going on with ChatGPT. It is not "applying the concepts of a poem" or checking meter and verse, it is generating text to fit a (admittedly very complicated) function that minimizes the statistical improbability of its appearance given the preceding text. One example is its use of rhyming words, despite having no concept of what words sound like, or what it is even like to hear a sound. It selects those words because when it has seen the word "poem" before in training data, it has often been followed by lines which happen to end in symbols that are commonly included in certain sets.

Human cognition is leagues different from this, as our symbolic representations are grounded in the world we occupy. A word is a representation of an imaginable sound as well as a concept. And beyond this, human intelligence not only consists of pattern-matching and replication but pattern-breaking, theory of mind, and maybe most importantly a 1-1 engagement with the world. What seems clear is that the robot was trained to recognize a certain pattern of pixels from a camera input, but neither the robot nor ChatGPT has any conception of what a "threat" entails, the stakes at hand, or the common-sense frame of reference to discern observed behaviors that are innocuous from those that are harmful. This allows a bunch of goofy grunts to easily best high-speed processors and fancy algorithms by identifying the gap between the model's symbolic representations and the actual world in which it's operating.

Also, it's not a very good poem. And it's definitions aren't entirely correct.

Which is a huge problem, because you cannot trust anything ChatGPT produces. It's basically an automated Wikipedia with an Eliza N.0 front end. Garbage in gets you garbage out.

We project intelligence whenever something appears to use words in a certain way, because our own training sets suggest that's a reliable implication.

But it's an illusion, just as Eliza was. For the reasons you state.

Eliza had no concept of anything much, and ChatGPT has no concept of meaning or correctness.

For now. This will change. And most humans do not write great poems either btw.

I tried that a few times, asking for "in the style of [band or musicians]" and the best I got was "generic gpt-speak" (for lack of a better term for it's "default" voice style) text that just included a quote from that artist... suggesting that it has a limited understanding of "in the style of" if it thinks a quote is sometimes a substitute, and is actually more of a very-comprehensive pattern-matching parrot after all. Even for Taylor Swift, where you'd think there's plenty of text to work from.

This matches with other examples I've seen of people either getting "confidently wrong" answers or being able to convince it that it's out of date on something it isn't.

Not sure how that's related. This is about a human adversary actively trying to defeat an AI. The roadway is about vehicles in general actively working together for the flow of traffic. They're not trying to destroy other vehicles. I'm certain any full self driving AI could be defeated easily by someone who wants to destroy the vehicle.

Saying "this won't work in this area that it was never designed to handle" and the answer will be "yes of course". That's true of any complex system, AI or not.

I don't think we're anywhere near a system where a vehicle actively defends itself against determined attackers. Even in sci-fi they don't do that (I, Robot movie).

"Saying "this won't work in this area that it was never designed to handle" and the answer will be "yes of course". That's true of any complex system, AI or not." This isn't about design, it's about what the system is able to learn. Humans were not designed to fly, but they can learn to fly planes (whether they're inside the plane or not).

The system wasn't designed to be able to learn either.

Reply on news.ycombinator.com