Google researchers deal a major blow to the theory AI about to outsmart humans | Hacker News Reader

Google researchers deal a major blow to the theory AI about to outsmart humans

businessinsider.com

26 pointsgardenfelder30 comments

All this talk about AI outstripping humans is beside the point.

The immediate risk is that unscrupulous corporations and governments will use it as a tool to further entrench their power. AI might or might not be seriously dangerous on its own in the far future, but it together with humans is going to be dangerous quite soon, if not already.

In fact, corporations have been using machine learning to expand their power for over a decade. The problem is not ai, but the lack of anti-trust enforcement.

The corporation are lobbying for "Make AI safe" monopolies. Also even the current chatbot style tech would be insanely good for a surveillance. Imagine AIs monitoring all the conversation for signs of dissent or tracking the population sentiments.

It seems historically, unscrupulous corporations and governments entrench their power by holding back technology from wider adoption or further development. As long as technology can be used by a significant portion of the populace, they all seem to lead to a more egalitarian society. I would hope AI is no different.

> unscrupulous

Exploitation for profit, expansion, and longevity is how capitalism works and as long as it is the system currently used, none of these actions are a surprise.

Change capitalism and we’ll change AI’s impact on humanity and the working class.

Dupe of https://news.ycombinator.com/item?id=38179200 but with standard BI clickbait.

Thank you. Business Insider is terrible. Pre-AI-Generated web pollution.

OpenAI/ScaleAI are basically trying to brute force AI by using massive fleets of human data labelers/writers to try and provide as much varied training data as possible. Will it work? Maybe, maybe not.

All I know is, they’ve shown impressive improvements year over year on cognition and generalization, and I have no reason to believe it’ll stop happening.

At risk of over-generalizing, the way you phrased this reminded me of the prior knowledgebase grand projects like Cyc.

As a fence-sitting skeptic, I sometimes wonder if all "solve it with scale" projects are just using different syntax to reproduce an analogue of the experiment we're continuously executing as a whole civilization. Can our joint artifacts and social systems understand something more than we do? Can they exhibit any emergent rationality that exceeds the participants?

Or are all these systems fundamentally a chaotic and contradictory mix? Is the understanding and rationality always an illusion, only coherent in localized extracts? Does it always need us (the homunculi) to monitor and steer it, fundamentally limited by the understanding of our best individual minds?

That's a highly inaccurate summary of what OpenAI and unsupervised LLMs do.

Yes there's human feedback at the end, but that's not to make it smarter or generalize facts better, it's so the AI isn't a genocidal dick. It's not about "labeling" correct facts for training.

No they did not. They tested GPT-2. GPT-2 is the version that gave us those classic AI Dungeon games that were absolutely hilarious because the model was so dumb that it couldn't follow from one end of a sentence to another.

I think that there are lots of cognitive abilities and characteristics that are missing from LLMs or could have better approaches. But looking at the performance of GPT-2 doesn't "deal a blow" to anything remotely near leading edge.

How has the transformer architecture changed from GPT2,3,4?

It doesn't prove anything about the capabilities of any model other than the one tested.

It's not just the most basic aspect of the architecture. The model weights determine the capabilities to a large degree.

I took a quick look into the paper. To be honest (I'm far from being an expert). As I understand it seems they took some toy models, and tried to show that the models do not generalize out of sample.

To me it is unclear what does it say about much more complex models? So for example there might be already so much structure, that you do not need to generalize out of sample and maybe humans don't do it as well.

agree, I am not sure how useful this is. Do humans generalize outside of their training data? Would you hire a divorce lawyer to work on an M&A transaction? nobody would do that. Would you hire a local butcher to do open heart surgery ?

Do humans generalize outside of their training data?

Absolutely not. Never. If that happened, you'd get programmers thinking they could solve every type of problem in the world, based on wildly oversimplified mental models. It would be utter chaos.

You ask: "Do humans generalize outside of their training data?" But then you give two examples of people specializing outside of their training data. But even then a divorce lawyer would do a lot better than me at M&A because they can generalize from what they learned about the law from divorce cases.

> Do humans generalize outside of their training data?

JFC, is this a joke?

> Do humans generalize outside of their training data?

Obviously they don't.

You jest, but every word coming out of our mouths is a generalisation that someone thought of.

Oh no, I was being serious.

People just don't understand how vast the unexplored in-distribution space is.

The paper

https://arxiv.org/abs/2311.00871

Yeah this is really silly IMO.

We know that transformers can generalize within the training set. We know that transformers can make connections between wildly different domains (at least when prompted).

Of course it can't generalize beyond training - why would it? But at the same time, there is probably huge amounts of value lurking INSIDE the training data that humans haven't unlocked yet.

> We know that transformers can generalize within the training set.

> Of course it can't generalize beyond training - why would it?

4 out of 5 people I discussed this subject didn't know, and even believe that current LLMs are bound within their training set. They claimed that LLMs could synthesize data beyond their training set, and the resulting answers will never be wrong.

There's a large misunderstanding about how these things work, and LLM developers do not spend the effort to fix this misunderstanding since it helps to raise the hype even further.

LLM's can, of course, synthesize data which is not in their training set.

Of course they can't create new facts, other than in principle, ones that can be derived from the training data.

I supposed generalization outside the training set structure may occur by chance should the outside set share enough of the same 'structure'. Basically if you can find magical maps to your training set then perhaps generalization may occur.

Ah yes, time again for Ximm's Law: every critique of AI assumes to some degree that contemporary implementations will not, or cannot, be improved upon.

Lemma: any statement about AI which uses the word "never" to preclude some feature from future realization is false.

Clickbait.

This seems a bit reductionist. It is hard to cut through the hype and counter hype but it can makes sense to reason about the theoretical limits of a technology. For a toy example we have something like the halting problem which was proven undecidable decades ago and this is still true despite computers getting way better. We use theoretical bounds all the time.

The term AI is nebulous enough such that new, unrelated, technologies can be invented and called AI but if you are talking about analyzing LLMs specifically you could probably figure out some meaningful theoretical bounds. This appears to be what these researchers are starting to try to do.

any model learned by gradient descent is equivalent to a kernel machine

*approximately

https://arxiv.org/abs/2012.00152

Reply on news.ycombinator.com