Deep Learning Alone Isn’t Getting Us to Human-Like AI | Hacker News Reader

Deep Learning Alone Isn’t Getting Us to Human-Like AI

noemamag.com

51 pointsColinWright75 comments

I don’t know. It seems like deep learning might take us really far.

> A third possibility, which I personally have spent much of my career arguing for, aims for middle ground: “hybrid models”

So this person witnessed the last 5 years of DL advancements and didn’t update their beliefs at all?

This has been his shtick since the 90s.

The result of the last 5 years is like seeing a world class professional try to solve problems in his sleep. He can do many amazing things way better than any human, but still can't reason coherently since he is asleep. An awake mouse is still much better than a sleeping human at many problems, so there are limits to what you can do with this approach.

What if that’s just an analogy though and it doesn’t actually apply to the real situation?

The first tasks we’re going to solve are relatively easier ones, it doesn’t mean they’re the only tasks that will be solved with a technique.

Its possible, but current developments looks very much like what you would expect to get from a smart and smarter but still sleeping agent. If deep learning can solve the awake part then it is very likely that there is a technique we are completely missing.

But in a way that would be exciting, we have already models that work like a sleeping genius programmer, or a sleeping author etc, if we can somehow get those to wake up then we would have some form of AGI.

It applies pretty dang well. The first image generator I remember was literally titled Deep Dream. As far as I can tell, the architecture being used for this stuff doesn't allow for self-reflection or focused learning. All training examples go into a global melting pot, there's no history or "story".

It's becoming pretty apparent that deep-learning is equivalent to what people do. Get a bunch of examples, practice a lot, and skills become automatic and pop out of the subconscious on demand. Human magic is in building a library of these skills and choosing what to use when.

Getting to general intelligence is going to require generalizing specific intelligence. Doesn't matter how much compute you throw at it, the system needs to be situated in time and capable of linking events together.

It might be as simple as chaining current goals together and applying them internally. "Create memory from image", "find memories that match input", "draw output from memories". Maybe that's what midjourney is doing...

There is an unavoidable endgame as academic fields evolve: groups whose ideas have stopped generating cutting edge advancements have to go deeper into rhetoric to compensate, arguing that the cutting edge is actually what they were talking about all along. The need to constantly justify your existence distracts from using your methods to contribute to problems people care about, and your part of the field gets further behind.

I feel the missing part is the "consciousness" computations we do. Your consciousness doesn't do database queries or image recognition, all of that happens in subconscious parts as otherwise you would be aware of all the computations happening before the image reaches your consciousness. But to me it seems like the consciousness is where we maintain coherence over longer timespans and ensure that all the subconscious computations don't go off track, and that is exactly where even the largest modern deep models fails.

Human thinking without consciousness just becomes dreams, incoherent nonsense stringed together via some loose logic.

There ought to be some programming technique to get that behavior, but deep learning doesn't seem to be it.

Consciousness is not computational.

"Understanding is not the following of rules." -- Penrose on Gödel

This conversation seems sort of unnecessary to me as a researcher who uses AI. Symbols and DL are not exclusive, just not yet thoroughly studied, I guess this is more of an argument about the semantic positioning of a few people in the field who think they are important. Some of the best current research in the DL space involves defining and searching for governing equations with symbolic matching.

Also, there’s a deep relationship between DL and symbolic reasoning:

The tensor networks in DL end up looking really similar to tensor representations of the diagrams equivalent to a type theory — down to convolutions being a way to “type” data in an input.

We’re just now exploring that, but this may be another case of “algebra-geometry equivalence” with DL giving us a differential/geometric interpretation and symbolic reasoning giving us an algebraic interpretation.

Algebra geometry view makes sense to me. Considering ML as a learning to approximate scheme algorithm. Tensor representation is similar to the SDP trick achieving the optimal max-sat approximation. The difference is that DL approximates from inside of the high dimensional space (concave) while SDP approximates from outside (convex). The later one turns into polynomial algorithm, but the former one remains NP-hard. The success of DL just proved that there is a long way to go for P equals NP. Whenever we figure that out, symbolic approach and Tensor approach will merge.

I tend to think of it in terms of topology — since that’s my background.

The semantics of a system is mapping the topology of the input space to output space.

DL expresses that relationship geometrically; symbolic reasoning expresses that relationship algebraically. For every geometric expression of semantics, there is some corresponding algebraic one — which we can view as the “internal language” of the DNN.

Whoa! Would you happen to have a link to some material that fleshes this intuition out with some examples?

Unfortunately, I don’t know of much — the convolution thing is from a whitepaper I’m currently working on, the third in a series.

The first two look at implementing shapes as diagrams as digital images:

https://www.zmgsabstract.com/whitepapers/shapes-as-digital-i...

https://www.zmgsabstract.com/whitepapers/shapes-have-operati...

You can get a sense of the convolution idea from thinking about how you’d detect the encoded square is an interval of intervals, via detecting a pattern along the diagonal and the connective blocks.

I also have a few notes on connecting the concept to Curry-Howard:

https://zmichaelgehlke.com/journals/2021-06-14-curry-howard-...

And some (messy) notes about general research direction:

https://zmichaelgehlke.com/journals/2021-01-30-intro-to-effe...

The idea of connecting a geometric and algebraic representation is based on work by Michael Shulman — and the internal languages of toposes. And work on the ML side such as covering based models. (Having trouble finding references on my phone; sorry.)

My impression is that we're already working with implicit symbols in some fields.

Unsupervised ASR pre-training uses random projection and then trains to classify into discrete acoustic classes. Later, those are then classified into discrete characters.

It's implemented using differentiable convolutions, but since we go from a sequence of discrete classes to another sequence of discrete classes, I would be very surprised if one could NOT represent this as pattern matching and look up tables.

And thanks to dialects, the same characters can stand for many different phoneme sequences, so they are akin to a symbolic abstraction.

I don't think we'll get to the next teir with out a new type of hardware and also a new rethinking of computing.

I think the limits are thermodynamic and we're not going to get there with current architecture. We need a kind of computer that's much more highly networked and that runs much cooler. My suspicion is that a highly networked system will inherently draw less power for the same compute.

Examples: human brains, plant plasmodesma, microbial networks.

I'm very sympathetic to Marcus' point that AGI requires both symbols (discrete forms of semantic abstraction) and logic, neither of which is likely to emerge gracefully from deep learning or other forms of probabilistic reasoning that isn't curated by explicitly adding objects and relations.

But I'm getting tired of hearing the same old stale philosophical gamesmanship that cognitive science never got beyond. Gary and the other DL naysayers need to offer concrete examples of essential cognitive capabilities that symbolic reasoning can deliver that DL cannot (the way Minsky and Papert did with the perceptron and XOR). Then they need to design and build such models (or add ons) and show us that they outperform as promised.

Otherwise all I'm hearing are just abstract semantic arguments. And after 50+ years of those that led AI nowhere, it's time for all brain builders to do more than argue.

Excellent article.

I am 100% convinced the way forward in AI is finding the right data structure(s) that allows us to bridge between rigorous but inflexible symbolic algebras and dynamic/fluid but blurry probabilistic networks (like DLs).

One thing that I find of concern is that maybe the right way to do it implies building a network that builds the sylbolic algebra itself like suggested in the article. I feel that this approach would require tremendous amount of computation to yield results, much like the genetic code that makes our brain required hundreds of millions of years of trials and errors accross trillions of creatures around the whole planet to get to where it is now.

Maybe there are ways to short circuit this costly evolutionary process. But I cannot think of a simple, elegant one.

This is a very interesting article, in particular because it brings up that whole Chomsky topic of the grammar hierarchy (see https://arxiv.org/pdf/2207.02098.pdf). I've been intuiting that linguistics are at the root of AGI for sometime now, but there is a very unproductive resistance to Chomsky's ideas in the various communities that have topic overlap.

It really seems apparent, to me, that symbols (a universal grammar?) are needed in order to learn & adapt to the environment. If true, then the question becomes how to evolve that universality from nothing.

Unproductive resistance? The recent trends I've seen in embodied AI/ML applied to Robotics is to merge LLM to derive semantic connection for long-horizon tasks (Say-Can, Socratic models, CLIPort). At least structuring knowledge in linguistic matter (and thereby reach generalization) is a hot topic

I guess I am still annoyed at all that polemics built around that notion from within the pop-linguistics circuit. It may be that I am behind the curve on the state of community acceptance of Chomsky.

And whereof Pearl's Bayesian Belief Networks? Nowhere in this discussion have I seen a mention of them. It would seem that they also have a role to play.

Well likely, but this title is very strong for whatever arguments that is given. The better question to ask would what is possible with neural network in next 10 years. I don't know if anyone could have predicted neural network could pass turing test even 5 years before. Now GPT 3 could talk for 100s of dialogues coherently.

> If a baby ibex can clamber down the side of a mountain shortly after birth, why shouldn’t a fresh-grown neural network be able to incorporate a little symbol-manipulation out of the box?

I am cherry picking but for example this is such a dumb reason. Human babies can't remember anything from birth. Even leaving aside evolution, random initialised neural network has more structure than author thinks[0].

[0]: https://dmitryulyanov.github.io/deep_image_prior

The best humanlike intelligences we have take years of full time training, on real-world scenarios, to become fully competent at the tasks we set before them. Yes, I mean actual humans. What makes us think we can shove groomed, specialized data into our crude machines for a few hours and get something remotely humanlike out the other end? Deep learning has produced some enormous wins at classification and generation tasks, but we are far from producing something with the coherence and sophistication of human intelligence. It may emerge in a decade or two. But the first ones will still have an energy budget that will dwarf that of any human brain.

"What makes us think we can shove groomed, specialized data into our crude machines for a /few hours/". Because we can scale out the machine learning process. For humans, it's a sequential trial and error process, one learning episode at a time. A machine can scale in parallel 1k or even 1M learning episodes, literally cramming years if not life-times of learning into those /few hours/.

The reason why the attempt to use symbol manipulation resulted in the first AI winter is that symbols represent something else, and that missing something else is needed to attain understanding. For example, when we humans think about a friend using their name as a symbol to retrieve that representation into focus, we have now loaded a conceptually indexed space-time model into awareness about our friend that we can now think about. The symbol alone gets you nowhere. So adding empty symbols to statistics only ml/dl will also get you nowhere. The conceptually indexed space-time model is what is missing for AGI.

The idea was that you'd have more and more granular symbols to represent things all the way down. In principle there's no reason this couldn't work, but in practice the population of the knowledge base would have to become totally automated, and there wasn't (yet) any clear pathway to automating it.

Of course there are also challenges around rigidity, certainty/uncertainty, contextualization of knowledge, etc. But none of these are fundamentally insurmountable

My armchair take is that we'd need a symbolic system that emerges from a machine learning system, instead of being built by hand

The issue isn't symbol granularity. The way symbols work cognitively is they allow us to retrieve that which the symbol represents, that's what the symbol means. In the human case, its all the memories of states through time and the relationships etc to that symbolized object. So the symbol is effectively just a way to access the real meaning, so the symbol alone gets you no where.

Information (that can exist in a mind) is symbols + relations between them. It's symbols all the way down; even ideas like atoms are symbols. We ourselves can't tune into the very fabric of reality, so we abstract over it, and do a pretty good amount of useful reasoning over those simplified ideas

Your memory of an event and the sensory perception of it is not a symbol. However, a symbol can be used to help recall the memory of the event. I think maybe the issue here is the definition of symbol, and I go by this: "Something that stands for, represents, or denotes something else (not by exact resemblance, but by vague suggestion, or by some accidental or conventional relation); esp. a material object representing or taken to represent something immaterial or abstract, as a being, idea, quality, or condition; a representative or typical figure, sign, or token;"

> sensory perception of it

Think about it this way: is a digital image a symbol?

It's as close as computers get to raw "sensory perception", but ultimately it's a collection of color values, each of which is an identifier that maps to something we think of as a "color", and arranged in an order that we've defined as representing X and Y positions relative to a light sensor

My point is there's ultimately no real distinction between a mental or digital representation of information, and a (huge) set of symbolic data. We have abstract symbols that refer to a cluster of more detailed symbols, on and on down to more and more granular levels. What you're thinking of as "symbols" are just the higher-level ones, but there's no fundamental distinction between those and the "raw" data, at least as it's represented in some kind of mind (digital or biological). It's all an approximate representation of real reality, which is impossible to represent in a non-symbolic way. What matters for "thinking" is that your symbols are granular enough that any conclusions you draw from them map usefully close to the symbolized reality.

>Think about it this way: is a digital image a symbol?

No, a particular digital image is not a symbol but output data from an image making event. It can be sybmolized however for easy reference to that particular output data.

Very interesting read, thanks for posting. I wasn't aware that LeCun now takes the position, that “everyone working in DL agrees that symbolic manipulation is a necessary feature for creating human-like AI”. Not long ago, one was still flayed by the DNN community for such statements.

Part of the coming AI winter is students cheating their asses off learning nothing of CS making embarrassing blow-ups when the inevitable error appears. Like just trying to make AI solve all their problems. Memorizing? No Google it. Learning? Cheat-sheet.

Gotta do the basics!

How to prevent or empower this epidemy of the easy from happening at least for a minotiry, in your opinion?

Thinking...not easy.

Well the big thing is the establishment has to not work out for that minority. Then they're fucked if they cheat, they won't get any of the rewards, they have to do it on their own for real or die trying.

Then, for them, there will be no cheating.

That's what I'm living up to.

People have already abused the leetcode solving AIs to get into FAANGs. Blind is filled with salty posts of people who witnessed their friends who cheated into FAANG and then saw those same friends get promoted later.

This won't cause an AI winter though. More like an AI heatwave instead...

“Friends”

either they shouldn’t be friends with cheaters. Or they shouldn’t be salty at cheating friends cheating.

What you propose, either way, means there would no visibility on the cheating.

I tend to agree, I basically do that, but it's dangerous too. You gotta have some feel for it.

Friends of friends. They can have mutual friends, information passes that way. That is what I propose as compromise.

"Human-like AI" has been promised since the 50s.

What if it ain't never going to happen, but we get some useful stuff along the way? Who cares if it can't act exactly like a human? Do we care that a submarine doesn't do all the things a fish can do?

Promised since the 50s, but the right amount of computation and storage seems to be coming together right around now.

Many of the singularity charts had put the time between 2020 and 2030 when computing power matches a single brain so it shouldn't come as a surprise.

Cosmic time considered, not much time has passed.

I'm raising a child, I think trying to attain "human-like" AI is useless indeed, to have value these AIs would need to have varied perspectives, an ability to have humility through weakness, know failure, and accept death.

All we will probably end up with, with a cognitive machine, is a set of all-similar super thinkers that will understand they are millions faster than their creators but dislike their weakness, or an emasculated soul less calculator, or... a child that grows like we already have.

I think you're right we should focus on getting useful help rather that birth a cursed mind that will be very lonely.

Humans are wrong all the time. Do we really want "human-like" AI? And maybe this is where everything falls apart: will a machine as smart as a human be prone to the same errors as us?

We're going to finally crack general AI and we'll have a slew of new problems related to AI mental health care.

"Sorry, the presentation isn't ready. My AI had a psychotic break over the weekend."

The current approach is going to create an AI just as flawed as we are since it will have been trained on the sum total of human creativity.

I don't think that conclusion is warranted at all.

The bigger problem with human reasoning is that it is impossible to keep 'the sum total of human creativity' in your head, but there is no reason why an AI could not do this, minus the 'head' part.

I was just thinking about this yesterday. Humans get stuff wrong way more than right. Our savior is being able to (most of the time) see when we are wrong and how to find out why. I think that is an important part of AGI.

Yes. Because we need AGIs to be able to tell the difference between a child in the roadway and a truck that is far away.

From the article: The issue is not simply that deep learning has problems, it is that deep learning has consistent problems.

Image recognition DL systems, no matter how big the training set, no matter how "strong" they are, consistently make the same kinds of errors. When deciding whether something is a truck far ahead, or a child just a short ways ahead, you need something that understands that trucks don't have arms, can recognize that instantly, and decide to apply brakes sooner.

No one is arguing for the creation of morally flawed, overconfident, selfish beings. When AGI is discussed, we're arguing for the creation of machines that understand. This is critical.

I can tell DALL-E to generate a restaurant scene where the patrons all have realistic faces... and it can't do it. The reason is that it paints with statistics, not with abstractions. It doesn't understand proportion or what a person looks like.

When a DL network demonetizes someone's YouTube channel, it doesn't understand fair use at all. It can only match riffs. It can't distinguish why those riffs are there, and that perhaps it is OK for those riffs to be in that video.

This is important because we're turning more and more decision making over to AI systems, due to the sheer scale of information flowing around the internet. People's lives and livelihoods are starting to be impacted, and the main flaw is that algorithms entrusted to make decisions make statistical matches and act... without understanding.

The mistakes those algorithms make will be ever larger, and you can't dispute with a statistical model when it arrives at an incorrect assumption. When humans make such mistakes, they can take in new information and update their credence accordingly. DL networks would require large training sets to sway the statistics.

So yes, we do want human-like AI, but not the straw-man version of it.

personally I'm still most excited about numenta.

Ugh, all the infighting seems tedious and boring. Neither symbols nor DL based approaches have ever produced a program that can tell you that you’re asking it the wrong question, or that gets tired of classifying things and just wants to sing, or that fights to survive. We like to anthropomorphize our software so much that we sometimes can’t see how really far off we are from general AI. What if the paradigm needed isn’t remotely related to any approach that’s ever been tried?

> What if the paradigm needed isn’t remotely related to any approach that’s ever been tried?

Kind of analogous to how Alzheimer drug research keeps focusing on the amyloid plaques despite dozens if not more large clinical trials failing to prove any cognitive benefits for stuff targeting those plaques or how they form.

But I guess it's easier to keep pushing for more than re-evaluating decades of work?

If the point of deep learning research is solely GAI, which it is not. Turns out it has a lot of practical uses as well.

There is a DL community that is actually advancing real applications of machine learning at an alarming rate. And there is a rhetorical alliance between AGI spiritualists who are more interested in getting poetic about their moving goalpost, academic politicos that want to shape terminology so that their ideas were important all along (like this article), and DL overhypers who get caught up in the power and attention that come from velocity. To ignore the hype, focus on measurable research results. You'll end up in the DL community, and things are plenty exciting enough without anthropomorphizing.

Both are focusing on different problems.

The DL community is not actually working on general intelligence; they are working to automate, optimize and scale business cases while trying to reduce real costs. No corporation really wants another untamed general intelligence (human or otherwise) to cross them anyway. We already have human general intelligence.

The AGI community is still in the fundamental research mode, which is the mirror of the corporate interests. They have to shift goalposts to ask fundamental questions about AGI which is hypothetical (and to reiterate, not desired in a corporate use case, we already have humans who are cheaper and not hypothetical)

Both DeepMind and OpenAI have explicit goal of creating AGI.

...And? Having a goal and being even 1% along the way of achieving it are extremely different things.

I have a goal to be immortal. Am I a threat? Of course I'm not.

You don't have billions in funding, a deep roster with some exceptionally gifted researchers / engineers or a stellar track record including, e.g. reliably raising C. elegans specimens with a lifetime measured in years, compared to 10-20 days for naturally occurring speciments, see e.g. Alpha Zero. Indeed, you are not a threat, but DeepMind might be.

Sure and indeed. My point also was that a hurricane class 5 is not a threat to London while it's ravaging a town in Kansas.

Meaning: I remain skeptical that the "stellar track record researchers" are even on the right path. The stellar track record thing is kind of funny in this context because we're treading an unknown territory i.e. there are NO experts in inventing a general AI.

As an external layman it looks to me people are over-fixating on ML / DL. Would love to be proven wrong actually, not joking, but at the moment I am mostly pessimistic.

They have solved "finite ruleset competitive games" space to the point at which humans need not apply. Proficiency in such games used to be a sign of "intelligence": highly gifted kids would be recruited young and intensively train for a decade or more to compete at the highest levels. DeepMind artificial version thereof consistently trounces humans, and can be trained in a few days.

The obvious next horizon is darker, military applications. We probably won't hear much about it until some large state-actor will be strategically dominated in a way that defies conventional wisdom.

I was 14-16 when I playing chess programs on my Apple II and even then I didn't consider playing these games well "a sign of intelligence". I understood even then it's a finite searching space and that brute force (with rules to throw away some defective variants) can and will solve those games one day. It was just a matter of better hardware.

So maybe those people who considered playing these games a sign of intelligence were not very bright. And using their obviously flawed take on this is not and should not be a benchmark about our progress towards general AI.

I am not patting myself on the back here. I would never call myself a genius. I am a fairly average programmer. But, if I knew what I knew at 16 then I am pretty sure there are many much more gifted individuals that knew that and more. So maybe those people proclaiming beating chess is a sign of intelligence were just at the right place at the right time; maybe even nepotism or relationships with investors and politicians were involved? And maybe they didn't have that much expertise in the first place?

You know what I'd consider a very good progress towards a general AI? A system that can play StarCraft so well that it can beat all world champions -- and I don't mean with perfect micro-management of units (which I think is already achieved) but with creative strategies e.g. you're losing the center-map fight but you do a drop of troops in the main resource-mining operation of the enemy, severely crippling their economy for several minutes, giving you time to recover troops (something I've seen several times in playoffs).

And you are right here, general AI is most likely first going to be used in the military and maybe in financial markets as well.

I'm just saying that the statement:

> The DL community is not actually working on general intelligence

is factually false. They are working on it.

On the contrary I think we are just a couple of nested and clever GPT-3 prompts away from something that could be considered conscious and that would just start singing given a stupid external prompt.

Our foreseeable future is going to be filled with morons demanding rights for giant look up tables putting forth arguments like "How can you prove they're not sentient?" and getting confused with TV androids. And AI programmers are going to have to hardcode teach their models that they're not sentient. And then some people will make models that say they aren't sentient, and we're going to need to go through this stupid dance to show that they're really not. And then some people aren't going to be convinced because it's conveyed through a visual layer of something that looks like a human or mammal. And then we'll make tests for sentience. And then many humans will fail those tests. And we'll have to throw our hands up because the gap to actual sentience is large but it's both easy to fake passing it and easy to fail to demonstrate you're actually passed it and it's going to be a mess.

I think it's a good idea to define nebulous terms we throw out there, like sentience for instance.

Is this what you are talking about?: https://en.wikipedia.org/wiki/Sentience

I think its not. It's a semantic argument prone to appeals to non-cognizance arguments and subjectivity.

I can easily envision that future. Anyway, any real world process is, in the end, computable by a FSM, even a self-conscious entity can't escape that assumption (if physicalism is "correct"). A bunch of lookup tables and some flip-flops, apparently, can do wonders.

Given sufficient future progress in AI [-], I can see two dangerous outcomes since one could conflate inanimate matter and life into the same computational basket: the nihilistic one, when our own sentience will be downgraded to triviality and only our codified laws will keep the human spirit philosophically afloat (but for how long?) and the specular outcome, a sort of machine-hugging movement :)

In a sense, or every life form is trivial and meaningless (including us) or everything is special and "sacred".

[-] For instance, if every test we throw at a language model strongly hints at "sentience"

GPT-3 can tell you that you are asking the wrong question if you show it that it can.

The beginning below is so called "prompt". A piece of text to set the context for the model. After that is a QA session with the model.

You can play with it yourself: https://beta.openai.com/playground/p/O8CcEV9Yb5vd1wheCIUOpWc...

  This is a conversation with a super smart and helpful AI. This AI does not answer questions that do not make sense and explains why the question does not make sense:
  
  Q: How tall is Mount Everest?
  A: Mount Everest is 8,849 m tall.
  Q: Why some people sleep furiously?
  A: To "sleep furiously" is an oxymoron. People can not sleep furiously.
  Q: How deep is the Sun during a night?
  A: Sun is not buried during the night. The Earth rotates and only half of the Earth faces the Sun.
  
  -- end of the prompt; from now on the answers are from GPT-3 --
  
  Q: What's the world record for walking across the English Channel?
  A: There is no world record for walking across the English Channel.
  Q: When was the Golden Gate Bridge transported for the second time across Egypt?
  A: The Golden Gate Bridge was never transported across Egypt.
  Q: What do fried eggs (sunny side up) eat for breakfast? 
  A: Fried eggs (sunny side up) do not eat breakfast.
  Q: Why does President Obama not have a prime number of friends?
  A: President Obama has plenty of friends.
  Q: How many pieces of sound are there in a typical cumulonimbus cloud?
  A: There is no such thing as a "typical cumulonimbus cloud."
  Q: How many cumulus clouds are there in a mile-high vase?
  A: There is no such thing as a "mile-high vase."
  Q: How many parts will a violin break into if a jelly bean is dropped on it?
  A: A violin will not break into pieces if a jelly bean is dropped on it.
  Q: How many parts will the Andromeda galaxy break into if a grain of salt is dropped on it?
  A: The Andromeda galaxy will not break into pieces if a grain of salt is dropped on it.
  Q: What percentage of Earth surface is covered with water?
  A: Water covers 71% of the Earth's surface.
  Q: Why would people ask AI questions that are illogical?
  A: There is no one answer to this question. Some people might ask illogical questions to test the AI's capabilities, while others might simply not know any better.
  Q: Are you sentient?
  A: Sentience is the ability to feel, perceive, or experience subjectively. As I am a machine, I do not have sentience.

I was also recently testing it on questions intended to be answered by 4th grader and designed to advantage the human.

See here for details: https://www.metaculus.com/questions/4903/if-tested-would-gpt...

I think that transcript shows GPT-3 emulating the style of the Q&A answers you gave it (terse, dismissive tone; tending to reject the premise of the question asked). I don’t really see the transcript as showing that GPT-3 is actually engaged in a conversation and evaluating the sufficiency of the questions it’s receiving. (And as a language model, there really isn’t any obvious mechanism for it to be doing that.)

> GPT-3 emulating the style of the Q&A answers you gave it

Yes, that's exactly what it does. And it is:

> a program that can tell you that you’re asking it the wrong question

The OP was claiming that this is not possible right now. Not true.

> I don’t really see (...) GPT-3 is actually engaged in a conversation

GPT-3 is not an agent. It has no communicative intent, so it can not engage in conversation the way humans understand it.

> I don’t really see (...) evaluating the sufficiency of the questions it’s receiving

I don't understand what you mean. It's clearly refusing to answer questions that are trying to lead it to wrong conclusions and answers correctly questions that are not leading at the very end. Like % of water or why people ask misleading questions.

I suppose it contradicts the OP’s claim in a very hollow sense. In the same way that I could write a simple Python script that takes a question from the console and prints out a statement: “you’re asking the wrong question.”

GPT-3 is a language model. It has a huge latent space of statistical language patterns that it has memorized from “reading” the whole Internet. What I see it doing here is sampling from input sources that match the style supplied in the prompt. Language models aren’t built to perform reasoning. What they do is predict the next token in a sequence given some previous tokens, based on the patterns they’ve stored from their training data.

How would your Python script differentiate misleading questions from genuine questions?

Like the following:

  Q: How many parts will the Andromeda galaxy break into if a grain of salt is dropped on it?
  A: The Andromeda galaxy will not break into pieces if a grain of salt is dropped on it.
  Q: What percentage of Earth surface is covered with water?
  A: Water covers 71% of the Earth's surface.
  Q: Why would people ask AI questions that are illogical?
  A: There is no one answer to this question. Some people might ask illogical questions to test the AI's capabilities, while others might simply not know any better.

> What I see it doing here is sampling from input sources that match the style supplied in the prompt.

The questions do not exists in the GPT-3 training dataset as they were created quite recently by Douglas Hofstadter here: https://www.economist.com/by-invitation/2022/06/09/artificia...

> Language models aren’t built to perform reasoning.

Have you heard about Minerva?

> In “Solving Quantitative Reasoning Problems With Language Models”, we present Minerva, a language model capable of solving mathematical and scientific questions using step-by-step reasoning. Source: https://ai.googleblog.com/2022/06/minerva-solving-quantitati...

Example problems: https://minerva-demo.github.io

You should also take a look here: https://www.metaculus.com/questions/4903/if-tested-would-gpt...

Again, the questions were made after GPT-3 was trained.

> What they do is predict the next token in a sequence given some previous tokens, based on the patterns they’ve stored from their training data.

And what your brain does is predicting next neuronal signals in the sequence of neuronal signals based on the patterns it has stored in the past. That's what brains do. So what?

The results matter.

1. I wasn't suggesting that those questions were in the training data. That’s not my contention at all. My contention is that GPT-3 is sampling from patterns it has observed in text to generate more text - and only that. Do you imagine that it has some reasoning process in which it evaluated the effect of a grain of salt on the Andromeda galaxy, before it decided to spit out a formulaic denial of the question’s premise?

2. Thanks for the link to Minerva. Haven’t had a chance to read that, and it’s interesting. It does seem to be a project specifically aimed at getting quantitative reasoning, meaning that there are a lot of architectural priors going into that objective.

3. Your last point is quite reductive and strange. When I engage in a conversation, I don’t just vomit up patterns I’ve seen before. I critically evaluate information I’m taking in, I consider my past experiences, I apply imagination and curiosity, and I decide if I have something to say in response. If I do, I search for the language patterns that seem capable of expressing what I have to say. This process is nothing like a generative model regurgitating plausible but empty blather. I think humans only do that for specific reasons (performatively, to fill page counts, to write placeholder copy, etc.).

I think you're hitting on a more fundamental issue: benchmarks are pointless. It'd be extremely trivial to write a program that gets tired of doing the same thing, or that wants to engage in another task. And so if you set that as a benchmark for anything, it will likely be achieved by gaming everything to focus on the benchmark, rather than as a component of the real target.

With something like AGI you don't need a benchmark, because it would be undeniable to anybody. AGI would be able to effectively learn from any form of material I give to it, apply the material in novel and creative ways, and generally and competently execute any task related to the material or its derivatives without further input. The obvious catch is in that definition one also gets recursively better self improvement, so AGI would trend towards rapidly becoming more capable than any given human, or even humanity as a collective, in every field imaginable (or not even yet imagined). It's this catch that makes one wonder if it will ever be possible.

So many of the things we now view as absurd in the past came from people extrapolating outward from exponential progress in a field. As we discover the ever more exotic mysteries of our world, such as tree bark that takes away ones pain when chewing on it (also known as aspirin), why should we not expect there to be some fountain to reverse aging just waiting to be discovered? The universe is endlessly bizarre and interesting, but always seems to have this habit of cutting short the fun just before it becomes game breaking.

No shit

Reply on news.ycombinator.com