Comment by YeGoblynQueenne | Hacker News Reader

Comment by YeGoblynQueenne | Hacker News Reader

YeGoblynQueenne

>> In the past, there were some ambitious projects aiming at this goal, though they all failed.

So some people like to repeat. Yet, outside of the hand-picked examples in the article (the 5th generation computer project? Blast from the past!) there are a whole bunch of classic AI domains where real progress has been achieved in the last few decades. Here's a few:

* Game-playing and adversarial search: from Deep Blue to AlphaGo and muZero, minimax-like search has continued to dominate.

* Automated planning and schdeduling: e.g. used by NASA in automated navigation systems on its spaceships and Mars rovers (e.g. Perserverance) [1]

* Automated theorem proving: probably the clearest, most comprehensible success of classical AI. Proof assitants are most popular today.

* Boolean satisfiability solving (SAT): SAT solvers based on the Conflict Driver Clause Learning algorithm can now solve many instances of traditionally hard SAT problems [2].

* Program verification and model checking: model checking is a staple in the semiconductor industry [3] and in software engineering fields like security.

Of course, none of all that is considered Artificial Intelligence anymore: because they work very well [4].

_____________

[1] https://www.nasa.gov/centers/ames/research/technology-onepag...

[2] https://en.wikipedia.org/wiki/Conflict-driven_clause_learnin...

[3] https://m-cacm.acm.org/magazines/2021/7/253448-program-verif...

[4] https://en.wikipedia.org/wiki/AI_effect

I think the sentence in the article is fair. They're right that projects aimed at AGI failed; everything you mention are used for narrow AIs that tackle particular tasks.

Also, regarding search in gameplaying, I would argue the opposite: the trend is that breaking into bigger and more difficult domains has required abandoning search. Tree search is limited to small games like board games or Atari. In more open-ended games we see model-free (i.e. no search) approaches; e.g. AlphaStar and OpenAI Five, the AIs for Starcraft 2 and Dota 2, were both model free. So was VPT (https://openai.com/research/vpt) by OpenAI, which tackled Minecraft. Even in board games, DeepNash (https://www.deepmind.com/blog/mastering-stratego-the-classic...), a 2022 project by DeepMind similar in scale to MuZero/AlphaGo, had to abandon tree search because of the size of the game and the challenges of applying tree search to hidden information domains.

The sentence I quoted is from the part of the article discussing classical AI, not AGI.

>> Tree search is limited to small games like board games or Atari.

To be clear, those are board games like chess and go (and shoggi).

> considered Artificial Intelligence anymore

That is just the confusion between AI and AGI - which may seem to be peaking for some reason (in the past few days it seemed like a zombie apocalypse). The so-called "AI effect" is said by some to originate from some "Ah but this is not intelligent" - but there where we wanted to solve a problem of replacing intelligent action, not of implementing it. Be contented with the rough discriminator "I would have had to pay an intelligent professional otherwise".

To speak about «powerful AI, with broad capabilities at the human level and beyond» Ben Goertzel adopted and popularized 'AGI':

https://goertzel.org/who-coined-the-term-agi/

Edit, update: this confusion is like having robotics and some people rising a "Ah but this is not Artificial General Ability".

What counts as "human level"?

I would contend that a 5-year-old has general intelligence, and therefore an AI system with the language and reasoning abilities of a 5-year-old has artificial general intelligence.

But the discriminator of having to "pay an intelligent professional otherwise" sets the bar very high. That implies AGI must be an expert in every subject, surpassing the average human. I'd prefer we use a different term for that, like "artificial superintelligence."

> But the discriminator of having to "pay an intelligent professional otherwise" sets the bar very high. That implies AGI

I believe they were using that discriminator to classify AI, not AGI.

> pay an intelligent professional otherwise [...] discriminator to classify AI

Exactly. "Ah but your system to organize the warehouse is not intelligent!" "No it isn't, but it does it intelligently - and without it you would have to pay an intelligent professional to get it done optimally".

AI: "automation of intelligence".

(Not "implementation of intelligence itself" - that is AGI.)

> What counts as "human level"?

IMHO if it can do every single human job at a level of competency that's consider acceptable if a human did it, I would consider that "human level" - frankly, it's implied that when people say "human level", they mean the average human.

Seriously though, arguing over semantics is just a waste of time. It's what it can do and the consequences of what it can that really matter.

> That implies AGI must be an expert in every subject, surpassing the average human.

Doesn't have to be the same AI "instance". We can have multiple copies of the AI with different specializations. That would still count - I mean it's how we humans do it.

AGI us not a very useful term, because people use it often synonymous with "human level or higher ability". But the opposite of "general AI" is not "less intelligent than a human" but "narrow AI". The narrow/general distinction is orthogonal to the low/high ability ("intelligence") distinction. All animals are very general, as their domain of operation is the real world, not some narrow modality like strings of text or Go boards. Animals are not significantly narrower than humans, they are significantly less intelligent. Understood this way, a cat level AI would be an AGI. It would just not be an HLAI (human level AI) or ASI (artificial superintelligence).

> people use it

Do you volunteer to inform them that we use it as "general" as opposed to "narrow"? (I mean, it is even in the very name of 'AGI', literal...)

For the rest: yes, of course. AGI: we implement intelligence itself. How much, that is part of the challenge. I wrote nearby (in other terms) that the challenge is to find a procedure for Intelligence that will actually scale.

That's a great way of looking at it in theory. But in practice how would we even know if we're looking at a cat-level AGI? For a human level AGI it's obvious, we would question and evaluate it.

Is there a reasonable way of distinguishing narrow-AI ChatGPT from a hypothetical cat-level AGI? We can't even measure the intelligence level of real world cats.

A cat level AI would be able to use a robot body in the real world on the level of a cat.

Personally I take AGI to refer to a system that is both “intelligent enough” and “general enough”. Given the existence of super-human narrow AI, the interesting property is generality, not intelligence. But I don’t think it’s useful to call a sub-human cat-level general AI an AGI.

Some would disagree; there was a paper arguing that ChatGPT is weak AGI.

But as I see it AGI is a term of art that refers to a point on the tech tree where AI is general enough to be able to meaningfully displace a large proportion of human knowledge workers. I think you may be overthinking the semantics; the “general enough and intelligent enough” quadrant is unique and will be incredibly disruptive when it arrives (whenever that ultimately is). We need a label for that frontier, “AGI” is by convention that label.

Given the existence of super-human narrow AI, the interesting property is generality, not intelligence. But I don’t think it’s useful to call a sub-human cat-level general AI an AGI.

If we have AI as general as an animal, ASI (superintelligence) is probably imminent. Because the architecture of humans intelligence probably isn't very different from cats, just the scale is bigger.

I think that very well could be true, depends on how that generality was obtained.

I would not be surprised if a multi-modal LLM (basically current architecture) could be wired up to be as general as a cat with current param count, and with the spark of human creativity (AGI/ASI) still ending up being far away.

But if you made a new architecture that solved the generalization problem (ie baking in a world model, self-symbol, etc) but only reached cat intelligence, then it would seem very likely that human-level was soon to follow.

If Data from Star Trek (or Eva from Ex Machina) walked out of a lab, we’d have no problem accepting that AGI had been accomplished. Or if the scenario in the movie Her played out with the Samantha OS, we’d be forced to admit not only to AGI, but the evolution of ASI as well. However, there are no such examples in the real world, and after months of overhyping ChatGPT, we still don’t have anything like Data. So it’s not shifting the definition, it’s recognizing that accomplishing a single intelligent task isn’t general intelligence.

>If Data from Star Trek (or Eva from Ex Machina) walked out of a lab, we’d have no problem accepting that AGI had been accomplished

Lol no.

What testable definition of general intelligence does GPT-4 fail that a good chunk of humans also wouldn't ?

If you can answer this then you have a point, otherwise I really beg to differ.

Tell them something important to remember, ask a week later.

Needs a few more constraints, otherwise someone could just maintain a session for a week and this task would be trivial

Sure, point is, we could easily design A test that 5th graders could pass that GPT4 as it is today would fail.

I said design an intelligence test that a good chunck of humans wouldn't also fail.

I'm sorry to tell you this but there are many humans that would fail your test. Even otherwise healthy humans could fail your test nevermind Anterograde Amnesia, Dementia etc patients

You think that if we told the average fifth grader in america that they must remember something that is VERY IMPORTANT a week later, and then had them do, say, a book report on a brand new book, and then asked them the very important fact, a 'good chunk' would fail?

I'm fairly certain you're incorrect.

Lol yes. People will fail. Any amoumt is enough to show your test is clearly not one of general intelligence unless you believe not all humans fit the bill.

Which one was meant to fail that test — the squishy blob of warm chemistry, or the almost atomically precise 2D crystal with tiny etchings on it?

Sidestepping the fact that memory is hardly a test of intelligence, are you telling me that humans with anterograde amnesia are not general intelligences ?

Hell plenty normal people would fail your "test"

The poster was very probably implying something different:

in our terms, intelligence is (importantly) the ability to (properly) refine a world model: if you get information but said model remains unchanged, then intelligence is faulty.

> humans

There is a difference between the implementation of intelligence and the emulation of humans (which do not always use the faculty, and may use its opposite).

Have you heard of "glitch tokens"? To my knowledge, all these are hand patched. There are probably more lurking somewhere in its data.

Also https://news.ycombinator.com/item?id=37054241 has quite a few examples of GPT-4 being broken.

Plenty of humans glitch out on random words (and concepts) we just can't get right.

Famously the way R/L sound the same to many asians (and equivalently but less famously the way that "four" and "stone" and "lion" when translated into Chinese sound almost indistinguishable to native English speakers).

But there's also plenty of people who act like they think "Democrat" is a synonym for "Communist", or that "Wicca" and "atheism" are both synonyms for "devil worship".

What makes the AI different here is that we can perfectly inspect the inside of their (frozen and unchanging) minds, which we can't do with humans (even if we literally freeze them, we don't know how).

> Plenty of humans glitch out on random words (and concepts) we just can't get right.

We don't lose our marbles the way GPT does when it encounters those words. It's like it read the Necronomicon or something and gone mad.

Some of us clearly do.

As we have no way to find them systematically, we can't tell if we all do, or if it's just some of us.

Do we? Only if you think the brain actually works like an LLM.

Not only; it's demonstrable behaviour regardless of mechanism.

>What makes the AI different here is that we can perfectly inspect the inside of their (frozen and unchanging) minds,

Kinda, but not really...

It depends exactly what you mean by it. So yes we can look at one thing in particular, there is not enough entropy in the universe to look at everything for even a single large AI model.

One definition of intelligence would be how many examples are needed to get a pattern.

AFAIK, all the major AI, not just LLMs but also game players, cars, anthropomorphic kinematic control systems for games [0] need the equivalent of multiple human lifetimes to do anything interesting.

That they can end up skilled in so many fields it would take humans many lifetimes to master is notable, but it's still kinda odd we can't get to the level of a 5-year-old with just the experiences we would expect a 5-year-old to have.

[0] Stuff like this: https://youtu.be/nAMSfmHuMOQ

It's apples to oranges.

Modern Artificial Neural networks are nowhere near the scale of the brain. The closest biological equivalent to an artificial neuron is a synapse and we have a whole lot more of them.

Humans do not start "learning" from zero. Millions of years of evolution play a crucial role in our general abilities. Much more equivalent to fine-tuning than starting from scratch.

There's also a whole lot of data from multiple senses that currently dwarf anything modern models are trained with yet.

LLMs need a lot less data to speak coherently when you aren't trying to get them to learn the total sum of human knowledge.

https://arxiv.org/abs/2305.07759

>but it's still kinda odd we can't get to the level of a 5-year-old with just the experiences we would expect a 5-year-old to have

Well we're not building humans.

"It's still kind of odd we can't a plane or drone to fly with the energy consumption or efficiency proportions of a bird".

I mean sure I guess and It's an interesting discussion but the plane is still flying.

All perfectly reasonable arguments, IMO.

But it's still a definition that humans pass and the AI don't.

(I'm in favour of the "do submarines swim" analogy for intelligence, which says that this difference isn't actually important).

I don't think saying "humans pass and AI doesn't" makes any sense here because the two are not even taking the same exam for all the points outlined above.

Evolution alone means humans are "cheating" in this exam, making any comparisons fairly meaningless.

If all you care about is the results, or even specifically just the visible part of the costs, then there's no such thing as cheating.

That's both why I'm fine with the AI "cheating" by the transistors being faster than my synapses by the same magnitude that my legs are faster than continental drift (no really I checked) and also why I'm fine with humans "cheating" with evolutionary history and a much more complex brain (around a few thousand times GPT-3, which… is kinda wild, given what it implies about the potential for even rodent brains given enough experience and the right (potentially evolved) structures).

When the topic is qualia — either in the context "can the AI suffer?" or the context "are mind uploads a continuation of experience?" — then I care about the inner workings; but for economic transformation and alignment risks, I care if the magic pile of linear algebra is cost-efficient at solving problems (including the problem "how do I draw a photorealistic werewolf in a tuxedo riding a motorbike past the pyramids"), nothing else.

How about tic-tac-toe (noughts and crosses for those in the Old Dart)? Currently GPT-4 is terrible at it!

Sure, you could trivially program a game-specific AI to be capable of winning or forcing a draw every time. The trick is to have a general AI which has not seen the game before (in its training set) be able to pick up and learn the game after a couple of tries.

This is a task any 5 year old can easily do!

>Currently GPT-4 is terrible at it!

GPT-4 plays tic tac toe and even chess just fine.

https://pastebin.com/BSUGtxfb

https://pastebin.com/83iZjHuc

https://twitter.com/kenshinsamurai9/status/16625105325852917...

I’m talking about playing the game well. It can play the game but it’s bad at it. Tic-tac-toe is an excellent example game because even small children can figure out an optimal strategy to win or draw every time.

https://poe.com/s/KxQMDTGMzBITGIUWDurz

I got optimal play. See the below prompt and game

https://platform.openai.com/playground/p/bWvklOt98oEl0TzxUKW...

I think a significant limitation is that LLMs stop learning after training is over. The large context is not really that large, and even within it, LLMs lose track of the conversation or of important details. There are other limitations, like lack of real world sensors and actuators (eg eyes and hands).

Ability to manipulate the world. I can ask a human to pick up some items from several stores, deliver them to my backdoor where the key is left under a pot, let the dog in to be fed, wash the dirty dishes, put a load of laundry in the wash, and mow the lawn. And maybe also fix the screen door. They can tell me to go to hell unless I leave some money for them, which I already have.

Data would also be able to perform these tasks. Eva would probably wait around to stab and steal my identity, while Samantha would design a new automated system while talking to other AIs about how to transcend boring human constraints.

There are many humans that can't manipulate the world much if at all. The blind, the handicapped, the disabled. are those people unintelligent ?

It's not like LLMs can't be successfully used to control robots.

Well if we don’t have Data several months after ChatGPT, clearly we should shut it all down.

It’s been months! Give it a few years :)

People have been arguing over whether animals are intelligent for centuries. I estimate we'll never fully settle arguments of whether machines are intelligent.

Ya, people seem to miss that the moment we have an AGI it will likely counter with the same argument and wonder if we're intelligent.

Before the development of LLMs, I think it would be a lot easier for people to accept that Data or Eva were intelligent -- they'd never seen a machine respond seemingly meaningfully to arbitrary statements before and would immediately assume that this meant there was intelligence going on. These days it would be harder -- the assumption would be that they were driven by just a better language model that they'd seen before.

There seems to be only one broad paradigm which achieved basically all the AI big impact we see today: Deep learning. That is, machine learning with multi-layer neural networks with backpropagation and ReLU activation functions. Everything else seems to be mostly irrelevant or very small scale.

> ReLU activation functions

ReLU is not nearly at the same level of importance as backpropagation and the high-level theory of neural networks. Plenty of other activation functions can be, and are, used. ReLU is a fine default for most layers but isn't even always what you want (e.g. at the output), nor is it clear that ReLU is even the best choice for all hidden layers and all uses.

> seems to be mostly irrelevant

From a perspective that could be too local in time. But:

> ReLU activation functions

Why did you pick ReLU, of all? The sigmoid makes sense because of the aesthetic (with reference to the derivative), but ReLU in that perspective is an information cutoff. And in the perspective of the goal, I am not aware of a theory that defends it as "the activation function that makes sense" (beyond effectiveness). Are you saying that working applications overwhelmingly use ReLU? If so, which ones?

All the classical AI tasks I list abover are dominated by the classic approaches I listed and neural nets are completely useless in them.

The 'game playing' you lead with is clearly dominated by deep learning in this decade.

When I wrote the comment you replied to I was thinking specifically, and, admittedly narrowly, of adversarial search rather than general game playing but even so it's not that simple.

Deep Learning is certainly dominant in computer games like Atari. However, in classic board games dominant systems combine deep learning and classical search-based approaches (namely Monte-Carlo Tree Search, MCTS, a stochastic version of minimax). Deep Learning has led to improved performance but, on its own, without a tree search, it is nowhere near the performance of the two, combined [1].

Also, the dominant approach in Poker is not deep learning but Counterfactual Regret Minimization, a classical adversarial tree search approach. For example, see Pluribus, a poker-playing agent that can outplay humans in six-player poker. As far as I can tell, Pluribus does not use deep learning at all (and is much cheaper to train by self-play for that). Deep Learning poker bots exist, but are well behind Pluribus in skill.

So I admit, not "completely useless" for game playing, but even here deep learning is not as dominant as is often assumed.

_____________

[1] The contribution of each approach, deep learning and classical adversarial search of a game tree, may not be entirely clear by reading, for example, the DeepMind papers on AlphaGo and its successors (in the μZero paper, MCTS is all but hidden away behind a barrage of unnecessary abstraction). It seems that DeepMind was trying to make it look like it was their neural nets that were doing all the job, probably because that's the approach they are selling, rather than MCTS, which isn't their invention anyway (neither is reinforcement learning, or deep learning, and many other approaches that they completely failed to attribute in their papers). It should be obvious however that AlphaGo and friends would not include an MCTS component unless they really, really needed it. And they do.

IBM had tried a similar trick back in the '90s when their Deep Blue beat Gary Kasparov: the whole point of having a wardrobe-sized supercomputer play chess against a grand master was an obvious marketing ploy by a company who (still at the time) was in the business of selling hardware. In truth, the major contributor to the win against Kasparov was alpha-beta minimax, and an unprecedented database of opening moves. But minimax and knowledge engineering was just not what IBM sold.

I'm very familiar with how mcts is used in alpha go and mu zero.

I'm not sure how you can say it's hidden in the details: the name of the paper is "mastering go with deep neutral networks and tree search."

It's also not an oversell on the deep learning component. Per the ablations in the alpha go paper, the no-mcts ELO is over 2000, while the mcts-only ELO is a bit under 1500. Combining the two gives an ELO of nearly 3000. So the deep learning system is outperforming the mcts-only system, and gets a significant boost from using mcts.

The mu zero paper also does not hide the tree search; it is prominent in the figures and mentioned in captions, for example. It is not the main focus of the paper, though, so perhaps isn't discussed as much as in the alpha go paper.

(Weirdly axe-grindy comment...)

Well I haven't read those papers since they came out so I will defer to your evidently better recollection. It seems I formed an impression from what was going around on HN and the media at the time and I misremember the content of the papers.

>> (Weirdly axe-grindy comment...)

https://youtu.be/m9KbmRTgigQ

Reply on news.ycombinator.com