Hacker News Reader

Comment by dahart | Hacker News Reader

Ugh, all the infighting seems tedious and boring. Neither symbols nor DL based approaches have ever produced a program that can tell you that you’re asking it the wrong question, or that gets tired of classifying things and just wants to sing, or that fights to survive. We like to anthropomorphize our software so much that we sometimes can’t see how really far off we are from general AI. What if the paradigm needed isn’t remotely related to any approach that’s ever been tried?

View on HN

There is a DL community that is actually advancing real applications of machine learning at an alarming rate. And there is a rhetorical alliance between AGI spiritualists who are more interested in getting poetic about their moving goalpost, academic politicos that want to shape terminology so that their ideas were important all along (like this article), and DL overhypers who get caught up in the power and attention that come from velocity. To ignore the hype, focus on measurable research results. You'll end up in the DL community, and things are plenty exciting enough without anthropomorphizing.

Both are focusing on different problems.

The DL community is not actually working on general intelligence; they are working to automate, optimize and scale business cases while trying to reduce real costs. No corporation really wants another untamed general intelligence (human or otherwise) to cross them anyway. We already have human general intelligence.

The AGI community is still in the fundamental research mode, which is the mirror of the corporate interests. They have to shift goalposts to ask fundamental questions about AGI which is hypothetical (and to reiterate, not desired in a corporate use case, we already have humans who are cheaper and not hypothetical)

Both DeepMind and OpenAI have explicit goal of creating AGI.

...And? Having a goal and being even 1% along the way of achieving it are extremely different things.

I have a goal to be immortal. Am I a threat? Of course I'm not.

You don't have billions in funding, a deep roster with some exceptionally gifted researchers / engineers or a stellar track record including, e.g. reliably raising C. elegans specimens with a lifetime measured in years, compared to 10-20 days for naturally occurring speciments, see e.g. Alpha Zero. Indeed, you are not a threat, but DeepMind might be.

Sure and indeed. My point also was that a hurricane class 5 is not a threat to London while it's ravaging a town in Kansas.

Meaning: I remain skeptical that the "stellar track record researchers" are even on the right path. The stellar track record thing is kind of funny in this context because we're treading an unknown territory i.e. there are NO experts in inventing a general AI.

As an external layman it looks to me people are over-fixating on ML / DL. Would love to be proven wrong actually, not joking, but at the moment I am mostly pessimistic.

They have solved "finite ruleset competitive games" space to the point at which humans need not apply. Proficiency in such games used to be a sign of "intelligence": highly gifted kids would be recruited young and intensively train for a decade or more to compete at the highest levels. DeepMind artificial version thereof consistently trounces humans, and can be trained in a few days.

The obvious next horizon is darker, military applications. We probably won't hear much about it until some large state-actor will be strategically dominated in a way that defies conventional wisdom.

I was 14-16 when I playing chess programs on my Apple II and even then I didn't consider playing these games well "a sign of intelligence". I understood even then it's a finite searching space and that brute force (with rules to throw away some defective variants) can and will solve those games one day. It was just a matter of better hardware.

So maybe those people who considered playing these games a sign of intelligence were not very bright. And using their obviously flawed take on this is not and should not be a benchmark about our progress towards general AI.

I am not patting myself on the back here. I would never call myself a genius. I am a fairly average programmer. But, if I knew what I knew at 16 then I am pretty sure there are many much more gifted individuals that knew that and more. So maybe those people proclaiming beating chess is a sign of intelligence were just at the right place at the right time; maybe even nepotism or relationships with investors and politicians were involved? And maybe they didn't have that much expertise in the first place?

You know what I'd consider a very good progress towards a general AI? A system that can play StarCraft so well that it can beat all world champions -- and I don't mean with perfect micro-management of units (which I think is already achieved) but with creative strategies e.g. you're losing the center-map fight but you do a drop of troops in the main resource-mining operation of the enemy, severely crippling their economy for several minutes, giving you time to recover troops (something I've seen several times in playoffs).

And you are right here, general AI is most likely first going to be used in the military and maybe in financial markets as well.

I'm just saying that the statement:

> The DL community is not actually working on general intelligence

is factually false. They are working on it.

I think you're hitting on a more fundamental issue: benchmarks are pointless. It'd be extremely trivial to write a program that gets tired of doing the same thing, or that wants to engage in another task. And so if you set that as a benchmark for anything, it will likely be achieved by gaming everything to focus on the benchmark, rather than as a component of the real target.

With something like AGI you don't need a benchmark, because it would be undeniable to anybody. AGI would be able to effectively learn from any form of material I give to it, apply the material in novel and creative ways, and generally and competently execute any task related to the material or its derivatives without further input. The obvious catch is in that definition one also gets recursively better self improvement, so AGI would trend towards rapidly becoming more capable than any given human, or even humanity as a collective, in every field imaginable (or not even yet imagined). It's this catch that makes one wonder if it will ever be possible.

So many of the things we now view as absurd in the past came from people extrapolating outward from exponential progress in a field. As we discover the ever more exotic mysteries of our world, such as tree bark that takes away ones pain when chewing on it (also known as aspirin), why should we not expect there to be some fountain to reverse aging just waiting to be discovered? The universe is endlessly bizarre and interesting, but always seems to have this habit of cutting short the fun just before it becomes game breaking.

> What if the paradigm needed isn’t remotely related to any approach that’s ever been tried?

Kind of analogous to how Alzheimer drug research keeps focusing on the amyloid plaques despite dozens if not more large clinical trials failing to prove any cognitive benefits for stuff targeting those plaques or how they form.

But I guess it's easier to keep pushing for more than re-evaluating decades of work?

If the point of deep learning research is solely GAI, which it is not. Turns out it has a lot of practical uses as well.

On the contrary I think we are just a couple of nested and clever GPT-3 prompts away from something that could be considered conscious and that would just start singing given a stupid external prompt.

Our foreseeable future is going to be filled with morons demanding rights for giant look up tables putting forth arguments like "How can you prove they're not sentient?" and getting confused with TV androids. And AI programmers are going to have to hardcode teach their models that they're not sentient. And then some people will make models that say they aren't sentient, and we're going to need to go through this stupid dance to show that they're really not. And then some people aren't going to be convinced because it's conveyed through a visual layer of something that looks like a human or mammal. And then we'll make tests for sentience. And then many humans will fail those tests. And we'll have to throw our hands up because the gap to actual sentience is large but it's both easy to fake passing it and easy to fail to demonstrate you're actually passed it and it's going to be a mess.

I can easily envision that future. Anyway, any real world process is, in the end, computable by a FSM, even a self-conscious entity can't escape that assumption (if physicalism is "correct"). A bunch of lookup tables and some flip-flops, apparently, can do wonders.

Given sufficient future progress in AI [-], I can see two dangerous outcomes since one could conflate inanimate matter and life into the same computational basket: the nihilistic one, when our own sentience will be downgraded to triviality and only our codified laws will keep the human spirit philosophically afloat (but for how long?) and the specular outcome, a sort of machine-hugging movement :)

In a sense, or every life form is trivial and meaningless (including us) or everything is special and "sacred".

[-] For instance, if every test we throw at a language model strongly hints at "sentience"

I think it's a good idea to define nebulous terms we throw out there, like sentience for instance.

Is this what you are talking about?: https://en.wikipedia.org/wiki/Sentience

I think its not. It's a semantic argument prone to appeals to non-cognizance arguments and subjectivity.

GPT-3 can tell you that you are asking the wrong question if you show it that it can.

The beginning below is so called "prompt". A piece of text to set the context for the model. After that is a QA session with the model.

You can play with it yourself: https://beta.openai.com/playground/p/O8CcEV9Yb5vd1wheCIUOpWc...

  This is a conversation with a super smart and helpful AI. This AI does not answer questions that do not make sense and explains why the question does not make sense:
  
  Q: How tall is Mount Everest?
  A: Mount Everest is 8,849 m tall.
  Q: Why some people sleep furiously?
  A: To "sleep furiously" is an oxymoron. People can not sleep furiously.
  Q: How deep is the Sun during a night?
  A: Sun is not buried during the night. The Earth rotates and only half of the Earth faces the Sun.
  
  -- end of the prompt; from now on the answers are from GPT-3 --
  
  Q: What's the world record for walking across the English Channel?
  A: There is no world record for walking across the English Channel.
  Q: When was the Golden Gate Bridge transported for the second time across Egypt?
  A: The Golden Gate Bridge was never transported across Egypt.
  Q: What do fried eggs (sunny side up) eat for breakfast? 
  A: Fried eggs (sunny side up) do not eat breakfast.
  Q: Why does President Obama not have a prime number of friends?
  A: President Obama has plenty of friends.
  Q: How many pieces of sound are there in a typical cumulonimbus cloud?
  A: There is no such thing as a "typical cumulonimbus cloud."
  Q: How many cumulus clouds are there in a mile-high vase?
  A: There is no such thing as a "mile-high vase."
  Q: How many parts will a violin break into if a jelly bean is dropped on it?
  A: A violin will not break into pieces if a jelly bean is dropped on it.
  Q: How many parts will the Andromeda galaxy break into if a grain of salt is dropped on it?
  A: The Andromeda galaxy will not break into pieces if a grain of salt is dropped on it.
  Q: What percentage of Earth surface is covered with water?
  A: Water covers 71% of the Earth's surface.
  Q: Why would people ask AI questions that are illogical?
  A: There is no one answer to this question. Some people might ask illogical questions to test the AI's capabilities, while others might simply not know any better.
  Q: Are you sentient?
  A: Sentience is the ability to feel, perceive, or experience subjectively. As I am a machine, I do not have sentience.

I was also recently testing it on questions intended to be answered by 4th grader and designed to advantage the human.

See here for details: https://www.metaculus.com/questions/4903/if-tested-would-gpt...

I think that transcript shows GPT-3 emulating the style of the Q&A answers you gave it (terse, dismissive tone; tending to reject the premise of the question asked). I don’t really see the transcript as showing that GPT-3 is actually engaged in a conversation and evaluating the sufficiency of the questions it’s receiving. (And as a language model, there really isn’t any obvious mechanism for it to be doing that.)

> GPT-3 emulating the style of the Q&A answers you gave it

Yes, that's exactly what it does. And it is:

> a program that can tell you that you’re asking it the wrong question

The OP was claiming that this is not possible right now. Not true.

> I don’t really see (...) GPT-3 is actually engaged in a conversation

GPT-3 is not an agent. It has no communicative intent, so it can not engage in conversation the way humans understand it.

> I don’t really see (...) evaluating the sufficiency of the questions it’s receiving

I don't understand what you mean. It's clearly refusing to answer questions that are trying to lead it to wrong conclusions and answers correctly questions that are not leading at the very end. Like % of water or why people ask misleading questions.

I suppose it contradicts the OP’s claim in a very hollow sense. In the same way that I could write a simple Python script that takes a question from the console and prints out a statement: “you’re asking the wrong question.”

GPT-3 is a language model. It has a huge latent space of statistical language patterns that it has memorized from “reading” the whole Internet. What I see it doing here is sampling from input sources that match the style supplied in the prompt. Language models aren’t built to perform reasoning. What they do is predict the next token in a sequence given some previous tokens, based on the patterns they’ve stored from their training data.

How would your Python script differentiate misleading questions from genuine questions?

Like the following:

  Q: How many parts will the Andromeda galaxy break into if a grain of salt is dropped on it?
  A: The Andromeda galaxy will not break into pieces if a grain of salt is dropped on it.
  Q: What percentage of Earth surface is covered with water?
  A: Water covers 71% of the Earth's surface.
  Q: Why would people ask AI questions that are illogical?
  A: There is no one answer to this question. Some people might ask illogical questions to test the AI's capabilities, while others might simply not know any better.

> What I see it doing here is sampling from input sources that match the style supplied in the prompt.

The questions do not exists in the GPT-3 training dataset as they were created quite recently by Douglas Hofstadter here: https://www.economist.com/by-invitation/2022/06/09/artificia...

> Language models aren’t built to perform reasoning.

Have you heard about Minerva?

> In “Solving Quantitative Reasoning Problems With Language Models”, we present Minerva, a language model capable of solving mathematical and scientific questions using step-by-step reasoning. Source: https://ai.googleblog.com/2022/06/minerva-solving-quantitati...

Example problems: https://minerva-demo.github.io

You should also take a look here: https://www.metaculus.com/questions/4903/if-tested-would-gpt...

Again, the questions were made after GPT-3 was trained.

> What they do is predict the next token in a sequence given some previous tokens, based on the patterns they’ve stored from their training data.

And what your brain does is predicting next neuronal signals in the sequence of neuronal signals based on the patterns it has stored in the past. That's what brains do. So what?

The results matter.

1. I wasn't suggesting that those questions were in the training data. That’s not my contention at all. My contention is that GPT-3 is sampling from patterns it has observed in text to generate more text - and only that. Do you imagine that it has some reasoning process in which it evaluated the effect of a grain of salt on the Andromeda galaxy, before it decided to spit out a formulaic denial of the question’s premise?

2. Thanks for the link to Minerva. Haven’t had a chance to read that, and it’s interesting. It does seem to be a project specifically aimed at getting quantitative reasoning, meaning that there are a lot of architectural priors going into that objective.

3. Your last point is quite reductive and strange. When I engage in a conversation, I don’t just vomit up patterns I’ve seen before. I critically evaluate information I’m taking in, I consider my past experiences, I apply imagination and curiosity, and I decide if I have something to say in response. If I do, I search for the language patterns that seem capable of expressing what I have to say. This process is nothing like a generative model regurgitating plausible but empty blather. I think humans only do that for specific reasons (performatively, to fill page counts, to write placeholder copy, etc.).

Reply on news.ycombinator.com