Hacker News Reader

Comment by mcnichol | Hacker News Reader

I feel this is merely stating the obvious.

It goes back to the industrial revolution and every revolution before and after. There was a point where they thought a majority of people would be working as switchboard operators and then telephone switching came along. Horses to cars, etc.

Jobs transition, new space is created, people are caught in tbe crossfire of supply and demand curves. Incan agree the bullwhip of this one is going to be aggressive.

I think it is a mistake to bet on AI as a driver of meaningful growth in a commercial capacity in the current state. When the data fed into an AI model is based on real data you can get predictability. When the AI inputs start coming from the AI outputs the wobble is introduced which results inevitably into delirium.

I think the world at large is "learning on the job" what data scientists and others in ai have known for a long, several decades long, time have already known.

View on HN

> When the data fed into an AI model is based on real data you can get predictability. When the AI inputs start coming from the AI outputs the wobble is introduced which results inevitably into delirium.

This is asserted without evidence. There is absolutely no proof that AI requires humans in the loop to function or improve itself.

Compare it with Deepmind's go-playing program, AlphaGo, which mainly involved training a neural network on a large database of high-level human games. It defeated one of the top-ranked players in the world, Lee Sedol, but arguably it was drawing from human experience just like you described.

But it didn't stop there. After that, Deepmind developed AlphaGo Zero, a version that was trained exclusively through self-play, with no human feedback in the loop. That's what you would call "AI inputs coming from AI outputs" but it didn't have the result of “resulting in delirium”: instead, it became orders of magnitudes stronger than the original version (which it defeated in a 100 to 0 competition after 3 days of training).

This shows that AI can improve itself without having access to any human knowledge, and indeed transcend human performance by orders of magnitudes. There is absolutely nothing to suggest that general AI cannot improve itself the same way.

People who claim otherwise are engaging in wishful thinking; they just assert their conclusion, but have no rational arguments to back it up.

You have no evidence that self improvement can work at generalized tasks. Go is a simple game with a clear win condition, but deep strategy and near infinite permutations of how a game plays out. Winning a game of Go is a task well suited to machine learning.

One might say you are engaging in wishful thinking by believing it’ll just continue to work across all domains. The world is much bigger than a Go board.

Anyone who works on stuff, especially novel stuff, quickly exposes the weaknesses of LLMs.

> You have no evidence that self improvement can work at generalized tasks.

I never made that claim. I just said it's way too early to rule it out: there is no logical reason why AIs will (always) need to have a human in the loop, and we don't have enough experience with LLMs to know what their true limits are.

I referenced AlphaGo not because the game of Go is exactly like every other task AI might perform in the future, but because the evolution from AlphaGo (which was trained on human games) to AlphaGo Zero (which was not) shows that at least in certain domains, it's not only possible to take the human out of the loop, it can actually make AI perform better.

I'm not claiming this will definitely be possible in every other domain, but people who state it definitely won't be, are jumping the gun.

>>When the AI inputs start coming from the AI outputs the wobble is introduced which results inevitably into delirium.

>This is asserted without evidence. There is absolutely no proof that AI requires humans in the loop to function or improve itself.

Nope, it's actually fairly well established.

https://pmc.ncbi.nlm.nih.gov/articles/PMC11269175/

That paper has had a pretty turbulent reception and looks pretty conclusively wrong at this point.

It used an incorrect theoretical framing that assumed that data was being replaced rather than accumulated as a result of more training (see https://arxiv.org/abs/2404.01413 which explores this). This is incorrect because this simply isn't how real-world datasets are created via synthetic data generation (which generally accumulate more data over time rather than replace their data). As a result most of the theoretical results were invalid.

Empirical evidence has also cast a considerable amount of doubt on the paper. For example, Microsoft Phi-4 was an empirical test in specifically what happens if the majority of your training data is synthetic rather than human and it turns out that Phi-4 did significantly better than previous models which relied primarily on human data.

There's some nuance to all of this in how exactly you do this, but the original claims of the paper are looking really shaky at this point.

We've already seen it start and they had to actively work against it: https://openai.com/index/where-the-goblins-came-from/

That article explains that the quirk was introduced during supervised fine-tuning.

> We unknowingly gave particularly high rewards for metaphors with creatures

It was the human feedback that caused the bias, not a change in training data.

The key word in that paper is “indiscriminate”, as in:

> We find that _indiscriminate_ use of model-generated content in training causes irreversible defects in the resulting models

If you view AI training as lossy compression of their training data, then lossily compressing the same data repeatedly will result in data degredation; this is well known from other domains (try repeatedly compressing a JPEG image, for example).

That means it's extremely important that there is some content curation in the loop. But there is no reason to believe this content curation must be done by humans, or that it must exclude all AI-generated content by default.

For example, the recent LLM-generated disproof of the Jacobian conjecture would probably be beneficial to include in the training data, despite being the result of an LLM.

Both programs had programmed in to their routines the rules of go (and for Alpha Zero the rules of chess and shogi). The programs then followed an optimization program. Absent the extremely specific "rules of this game" set by a human none of the programs work.

Amusingly, all three games are in the game theory sense trivial because they provably have a correct solution (probably a draw for chess), but the search space is so large that it is unlikely we will ever collectively invest the resources to discover the answer.

AlphaZero works in simple and fully observable environments like board games.

RLVR for LLMs works for coding and math domains which can be verified by code.

AlphaZero, RLVR and other RL methods don't work for learning dextrous manipulation for robotics, for example. Learning in robotics depends on human data.

You are referring to gans which have complete understanding of the rules and an api interface to leverage interacting with the world.

While I think there could be an argument made here you are mixing apples and rocks if you believe ChatGPT is using gans.

I hate to be that person but you are showing that you are unable to differentiate between linear regressions, clustering, gradient descent, and generative adversarial networks. You are over your ski-tips and don't see itc

> new space is created

That seems to be the crux here. You think it will be, I (and a lot of other people) aren't sure it will. If new space for jobs are created, I am certain we'll be fine long term.

What do you think will happen if it isn't (I know you have strong evidence for and reasons to believe it will be, but just indulge the hypothetical)?

Yes that’s the crux of the risk - will new space be created

On one hand. We’ve never had a technological revolution that didn’t yield more jobs

On the other hand, Terminator

> We’ve never had a technological revolution that didn’t yield more jobs

a) How many technological revolutions do you count?

b) Of these, which directly created jobs vs in which did the job numbers eventually recover?

Did you know “computers” used to be people?

Before electronic computers, rooms full of workers, often women, did the math by hand. That job was decimated.

Before ATMs, people went to bank tellers.

Before mechanized looms, there were weavers. Look up where the word “Luddite” comes from.

Before industrial manufacturing, shoes were made by cobblers.

Elevators used to have human operators.

There’s never been a technological revolution that didn’t yield more jobs than it replaced

There is a theoretical limit, but we can't know when it will occur.

AI is different because "intelligence" is the last thing humans still do better than machines.

Which provider do you use for artificial compassion?

I wouldn’t consider that a given. I still think it’s up in the air whether AI is “different” this time around.

> What do you think will happen if it isn't

Since such a thing has never happened, I doubt anyone, even scholars of history and anthropology, know.

Capitalism probably won't work in a world of ultra abundance though.

I can indulge the hypothetical and I appreciate your openness to engage in both sides.

I think as you stated, human nature and history is where I am looking at this from. People have a tendency to be unable to sit and do nothing for extended periods of time. I believe we are purpose driven but purpose is pretty loosely defined here.

Indulging the hypothetical (and arguably realistic in some circumstances) that AI replaces a tier of repeatable tasks I believe it would work from the bottom up. Tasks that are extremely repeatable would probably be gone 1:1 and it would be a sliding ratio as complexity increased where having human intervention as a backup becomes justifiable.

This would create a squeeze on the labor force which would bring up the very real and fair question of "How is it that I am unable to survive daily life when 100 - 200 years ago none of this automation / technology existed."

I think this could drive a sort of re-examining of what exists RE: a very inefficient and flawed system with a large amount of people possessing a lot of anxiety and time on their hands. It would be like the Eye of Sauron made up of those who were caught in the crossfire.

On the other side of the haves and have-nots I can imagine there being a very Harrison Bergeron style mindset of trying to popularize pacification until the next great innovation just solves itself.

On the extreme end, I just cannot imagine euthanasia, genocide, or population control schemes. It feels like we have come so far. How could it be possible we regress that far? I could imagine myself not wanting to live through any part of that. This seems like a very extreme and existential point to reach. I am trying to keep my thoughts pinned closer to biological and historical drivers.

I think we are hormone driven chemistry sets. I believe we respond to stimuli. I find it hard to believe anyone at any level could shut off that drive at such a massive scale and accurately predict the other side.

I imagine you're not from anywhere near former fascism or communism.

What historical evidence do we have to believe that these newfound capabilities are going to be distributed equitably?

Yeah definitely not fascist and I believe that is a control tactic not necessarily an equitability thing.

I don't think we have any evidence that anything will be done fairly. In fact the idea of Capitalism in a "fair system" has been objectively broken as incumbents apply pressure on free markets to suppress competition.

I think all forms have proved they can be broken where they no longer are "playing by the rules".

I was entertaining the hypothetical. I think the biggest unknown I feel which the article gently touches on is the unknown of how deeply this is trusted to make decisions that could be life-changing.

Imagine the danger of an AI model built to scan and understand a state or national level of policy that would be beneficial to society. Not only does that create a vector of manipulation to government that could be blindly trusted but it opens a cyber attack vector from other nation states.

I could see someone coming from a place of good without realizing that these functions are somewhat opaque without ceilings/floors which can increase deviation to elicit hallucinations.

Whether it is intentionally misused or a road to hell paved with good intentions it does feel treacherous. I do think there are many many democratizing aspects it offers but how do we stop mixing the baby and the bath water when we can't tell the difference.

If AI would become better at cognitive work than humans, I don't really see what new jobs would be created.

Construction for data centers?

Well clearly. But the above comment was about jobs that didn't exist before AI that AI will lead to the creation of.

Tiktok arguably led to more datacenters being created, AI is not unique in this regard and it's silly to imply that datacenter jobs meaningfully contribute to employment on the same scale as the loss of jobs AI could potentially lead to.

Yeah. I didn't mean to imply that. I think it's going accelerate the concentration of wealth.

This only works until general-purpose robotics gets good enough!

There are certain jobs where the human connection fundamentally commands a premium (e.g. celebrities, sports players, the relational economy), but it's hard to imagine a world where this sector employs the majority of people. It would be a precarious existence as these jobs would be the first to go in a crisis such as a war, etc.

I don't think it "works" at all. I believe the only parties that get a net benefit out of all this are basically the ownership class. They'll get some rental fees out of it. And they'll get a more direct lever to silently manipulate public sentiment. And the working class gets poorer, and more hooked on the dopamine machine.

Ok, so we agree!

What's next for workers? How can we push for redistribution?

No idea. If people get uncomfortable enough to take up pitchforks, maybe something will happen, but we're a very long way from that.

Industrial transitions pushed more people from leg-work to hand-work and then brain-work. What is left once AI is better at that?

This is potentially much more upending since it would put hundreds of millions of people out of work and concentrate all wealth at literally a handful of individuals. We could lead this towards a beneficial scenario with proper taxation, but the individuals who stand to gain will do anything in their power to prevent this.

> I feel this is merely stating the obvious.

I think there is value in someone of Gates' stature stating the obvious, plainly.

Reply on news.ycombinator.com