Hacker News Reader

Comment by zeryx | Hacker News Reader

Well written, but it side steps the X-risk and even the employment/ capitalism failures looming on the horizon.

2 years ago I thought we were decades away from general purpose AI, this is coming from a guy who implemented transformer models on day 5. My time estimates have been proven very wrong.

I'm equally worried about the value of white collar labour dropping to near zero in my lifetime, and the ultimate centralization of power. The movie Elysium seems less and less science fiction every day.

I am happy politicians and think-tanks are taking this seriously, you should be too.

View on HN

Have they been proven wrong? What is the roadmap to get to general purpose AI, and what is the proof that we are close?

The domains where AI experts can beat domain experts is certainly growing, but I don't see how you can get from that to a claim of general AI. I certainly don't see how you can get there from the recent LLMs in particular which can't beat domain intermediates at anything.

The transformer model is one more incremental improvement that made the language problem tractable. This has only captured the public interest because the language problem is so much more flashy and easy to understand than something like protein folding.

AutoGPT style of bots, aka agent universe discovery systems are still obviously nascent - but as someone who generally leverages LLMs for nearly all of my work, I would certainly say they are general purpose.

I would be remiss in saying they are anywhere near average human capability in most areas, but I do worry that my estimation capability is off and that we're closer to the concave part of the S curve than the convex. It just takes a couple more breakthroughs in model/dataset design

So these bots, as they currently exist, are already suitable for:

* Predicting the physical geometry of proteins

* Playing Starcraft

* Playing Go

* Trading stocks

* Controlling motors to make a bipedal robot walk

* Analysing data from particle accelerators

* Analysing data from telescopes

* Detecting cancer in medical images

These are all things that AI can do today. Are you suggesting that we are near a paradigm shift where a single AI system will ve competent in all of these domains? And will further be simmilarly competent in novel domains for which it has not been designed?

Language is the process of thinking rendered into tangible medium. If you master language, you master cognition. Many researchers recognized long ago that NLP would be one of, if not the, core of AGI.

>The transformer model is one more incremental improvement that made the language problem tractable. This has only captured the public interest because the language problem is so much more flashy and easy to understand than something like protein folding.

It's not just about language. Language models can predict novel protein structures, no folding required. https://www.nature.com/articles/s41587-022-01618-2

>What is the roadmap to get to general purpose AI, and what is the proof that we are close?

What testable definition of General Intelligence does GPT-4 fail that a big chunk of humans don't also fail ?

> 2 years ago I thought we were decades away from general purpose AI...My time estimates have been proven very wrong.

What's your working definition of "general purpose AI"? What does it include? What does it not include?

Does the AI have to be smart enough not to very often state incorrect information? LLMs still are unimpressive to me from that front.

I asked ChatGPT to explain the usage of the term "Wonder Kid" in the popular Apple TV+ series Ted Lasso, and which character it's tied with. It gave me confidently-stated incorrect answers multiple times until I told it the answer. (At which point it stated the correct answer with an apology.)

This doesn't feel close to "general purpose AI". Feels like we're meeting the limits of LLMs and finding they're not a magic AI bullet.

It's as good as an intern today, there's definitely room to improve to avoid saying incorrect things - great work being done on providing negative samples during RLHF training; also some papers working on incorporating trust (not near my laptop so don't have links)

Textbooks are all you need is another extremely interesting area of focus.

Improved correctness and a general reduction in hallucinations I predict will end up hitting OSS models by EOY

How is that "as good as an intern?" Interns, for the most part, do not hallucinate and confidently lie on topics they know nothing about. Good interns tend to ask a lot of questions and quickly learn the cultural cues around the office, allowing them to fit in and become productive. LLMs are incapable of any of this.

John Schulman from OpenAI recently gave a talk about the hallucination and uncertainty issues [1]. He claimed that models already have information about their uncertainty but that it remains an open problem as to how to express that uncertainty in natural language. One of the big issues with trying to prevent the model from hallucinating is that you can err too much on the side of caution, causing the model to lie about things it actually does know the correct answer to.

[1] https://www.youtube.com/watch?v=hhiLw5Q_UFg

>do not hallucinate and confidently lie on topics they know nothing about.

Oh shit, we've developed digital salespeople...

That doesn’t sound too surprising since isn’t that model trained on data from 2021? It may have very little information about Ted Lasso. But to your larger point, I personally don’t think I would derive all my complacency from an example like that simply because models can be trained on highly relevant data if that’s what it takes. Plain-vanilla chatGPT probably can’t take over the job of every entry-level office worker, but specific models can probably be trained individually to be a medical receptionist, health insurance coder, auto insurance claims representative, welfare caseworker, Ted Lasso analyst, etc. In that way, it’s really not that different than a ditzy, green but highly teachable 18-year-old in terms of its potential to take over a lot of jobs. I don’t have a solution/opinion/comment on how that will play out, just making that observation.

That scenario is easy to explain. If it hasn’t been trained on the Ted Lasso script then it’s going to just be guessing and will likely be wrong. Training a GPT model to identify when it lacks enough information to answer is something researches are actively pursuing. In the mean time you could try pasting in a transcription of the episode in question and then asking.

I'm equally worried about the value of white collar labour dropping to near zero in my lifetime

(Shrug) If it turns out that humans have a higher purpose than doing a robot's job badly, I don't see the downside to letting the robots do the job. I'm all for capitalism, but it's an ideology, not a religion. If there's something better on the horizon, let's help it along.

ML looks like it is probably going to be the next step in human evolution by punctuated equilibrium. And it's about time.

People thought that for a long time, that we'd be able to persue higher minded activities such as art, business management, entrepreneurship and writing.

What we've discovered is that the first model breakthroughs have been able to largely demolish that dream. As a writer I can tell you that writing with GPT4 today provides me with superpowers that non- accelerated authors could come close to; and theres no plausible way to distinguish our two works.

I'm more worried about the coming depression / suicide spiral (which were already in by the way) because of human uselessness than I am of terminators coming to farm us.

if only it could be a positive spiral.. like a big increase in Amish and Hare Krisna farms

As a writer you are pursuing higher minded activities. You aren’t working in a factory or steering a plow behind a team of oxen. You aren’t illiterate. You even have free time outside of your already very refined employment to pursue even loftier goals of personal fulfillment. Sure, not everyone can say the same but the proportion of people in grinding poverty continues to shrink and is minuscule compared to even just 100 years ago let alone compared to before the industrial revolution.

Can you provide some tangible examples of existential risk materializing?

Here's a mundane example; AI is aligned and used without harming humans. If/when robotics catches up, we will be heading to a world without a need for humans (at least based on the world pops we have today).

Humans _hate_ being useless, and we were already running into surplus human issues last decade in certain parts of the world.

I foresee a depression/suicide spiral never before seen in our history unless we do something radical

Plenty of people are quite happy without doing economically valuable work. Maybe most. So this isn't an X-risk in the sense that humans might go extinct. Workaholics may remove themselves from the gene pool, but there'll be plenty of people happily chilling.

I disagree with the last part in your scenario. It's going to be waves of murder and anarchy, not depression and suicide

Most people that retire don't turn into murderers or anarchists. I find very interesting how people tend to picture the worst possible scenarios, maybe a lot of Movies or TV influence?

How about news? Look at what Paris looks like right now with a small minority of "aimless youths" and multiply that by 1000x

There will always be places with social revolts. Still a tiny tiny fraction of how it was in the past.

Watching news can distort your perspective very quickly.

So 'now' has a problem. We have no means to know if our social environment is actually stable or not. Currently we are in a long 'peace' after WWII. Maybe it will remain, and improving technology will improve our lives and and things will continue to remain stable. Or, we'll have further climate instability coupled with AI labor displacement and things will go to shit faster than they ever have in history.

The point is people that believe the world will be stable in the future are more apt to build a stable future. If everyone is watching the news and they believe the future will be unstable, then the future will become unstable as a self fulfilling prophecy.

Technology that's putting those folks out of work (and into rioting, etc.) will also provide the tools to repress them. I don't like it, but that's the way it seems to me. E.g. Hong Kong.

I think that's a very limited view.

A lot of people that retire from their careers pursue other interests that fulfill their needs, maybe it will be the era of amateur artists everywhere. Others prefer to manually do tasks that can be automated.

Others could engage in cooperation with robots because humans will always have creative desires.

Amature artists doing what? Trying to figure out why human greed is insatiable, therefore they have no place to live?

There are any number of problems that have to be solved together. Giving the capital class hyper powerful AI robots so they can own the world while everyone else suffers is an AI risk.

Are people's self worth really intricately tied to what they do for a living?

I mean I enjoy my work, for the most part anyway. But if I could spend all my time with my family and hobbies I see that as an improvement rather then something to be depressed about.

Reply on news.ycombinator.com