Hacker News Reader

Comment by ralusek | Hacker News Reader

Almost everything interesting about AI so far has been unexpected emergent behavior, and huge gains through minor insights. While I don't doubt that the current architecture is likely to have a current ceiling below that of peak human intelligence in certain dimensions, it's already surpassed it in some, and there are still gains to be made in others through things like synthetic data.

I also don't understand the claims that it doesn't generalize. I currently use it to solve problems that I can absolutely guarantee were not in its training set, and it generalizes well enough. I also think that one of the easiest ways to get it to generalize better would simply be through giving it synthetic data which demonstrates the process of generalizing.

It also seems foolish to extrapolate on what we have under the assumption that there won't be key insights/changes in architecture as we get to the limitations of synthetic data wins/multi-modal wins.

View on HN

I mentioned this to another commenter as well:

You might want to reconsider your stance on emergent abilities in LLMs considering the NeurIPS 2023 best paper winner is titled:

"Are Emergent Abilities of Large Language Models a Mirage?"

https://arxiv.org/abs/2304.15004 https://blog.neurips.cc/2023/12/11/announcing-the-neurips-20...

Papers which get accepted with honors are not necessarily more truthful than papers which have been rejected. Yann LeCunn goes on twitter like any other grad student around NeurIPS or ICML/ICMR and bitterly complains when one of his (many) papers is rejected. Whose more likely to be correct here? Yann LeCunn (the TOP nlp scholar in our field by citations, who does claim that most emergent capabilities are real in other papers), or a NeurIPS best paper winner? My bet is on Yann.

Also, consider that some work gets a lot of positivity not for the work itself, but for the people who wrote it. Timnit Gebaru's work was effectively ignored until she got famous for her spat with jeff dean at google. Her citations have exploded as a result, and I don't think that most in the field think that the "stochastic parrot" paper was especially good, and certainly not her other papers which include significant amounts of work dedicated to claiming that LLM training is really bad for the environment (despite a single jet taking AI researchers to conferences being worse for the environment than LLM training circa that paper being written was taking). Doesn't matter that the paper was wrong, it's now highly cited because you get brownie points for citing her work in grievance studies influenced subfields of AI.

Please at least read the paper before appealing to authority. It is a well designed set of experiments that clearly demonstrates that the notion of a "phase change" (rapid shift in capabilities) as a popularized by many people claiming emergence is actually a gradual improvement with more data.

But if you do want to appeal to Lecun as an authority, then maybe you'll accept that these (re)tweets that clearly indicate he finds the insights from the paper to be valid:

https://nitter.1d4.us/ylecun/status/1736479356917063847 https://nitter.1d4.us/rao2z/status/1736464000836309259 (retweeted)

As for Timnit, I think you have your timeline confused. Model cards are what put her on the map for most general NLP researchers, which predates her difficulties at Google.

2018: Model cards paper was put on arXiv https://arxiv.org/abs/1810.03993

2019: Major ML organizations start using model cards https://github.com/openai/gpt-2/blob/master/model_card.md

2020: Model cards become fairly standard https://blog.research.google/2020/07/introducing-model-card-...

Dec 2020: Timnit is let go from the ethics team at Google https://www.bbc.com/news/technology-55187611

EDIT:formatting

Yann LeCun through Meta is incentivized towards maximizing capital return based on local maxima. That is how all business works, there is not really a direct incentive to pushing boundaries beyond what can be immediately monetized.

Gebru and her "Stochastic Parrots" did a big disservice to AI safety turning the debate into a shit-show of identity politics. Now she has her own institute, it was a move up for her career. Her twitter spats with Yann LeCun were legendary. Literally sent him to educate himself and refused to debate him.

Latest research shows emergent behavior is illusory. It doesn't preclude future emergence but currently models show 0 emergent behavior.

To me the most interesting aspect of LLMs is the way that they reveal cognitive 0-days in humans.

The human race needs patches to cognitive firmware to deal with predictive text... Which is a fascinating revelation to me. Sure it's backed up by psych analysis for decades but it's interesting to watch it play out on such a large scale.

When a human makes a mistake it is a "cognitive 0-day" but when an LLM does something correctly it is "illusory"?

The cognitive 0-day is not the way that humans act like LLMs, it's the way humans anthropomorphize LLMs. It's the blind faith that LLMs do more than they do.

The illusion of emergence is fact not fiction. The cognitive biases exposed by stochastic parrots are fact not fiction.

That is no different than saying beauty is only real if it is 100% natural. A woman who wears makeup and colors her hair is just an illusion of beauty.

It is a philosophical argument to say that a machine isn't truly intelligent because it isn't using the same type of neural network as a human

Parent is saying that with something as sophisticated as intelligence it's not enough to say that if it behaves like a duck it's a duck (which is what your seem to be saying and which the parent calls a 0-day).

There are some really good bulshitters who have led smart people into deep trouble. These bulshitters behaved really like ducks but they weren't ducks. The duck test just isn't good enough.

The -1 day is where people say that because LLMs behave like humans then humans must be based on the same tech. I just wonder if these people have ever debugged a complex system only to discover that their initial model of how it worked was way off.

That is a new definition of intelligence that you are using. You are saying that even when something can outperform humans in the SAT or other tests of intelligence, it isn't actually intelligent due to it not being a carbon based lifeform

Outperforming on the SAT(a test that the model was trained on) doesn't seem like a marker of intelligence as much as it is of recall.

No. I gave the example of a bullshitter - who is usually a carbon based life form.

What about papers like these that suggest creation of task-oriented manifolds?

https://www.biorxiv.org/content/10.1101/764258v3

> I also don't understand the claims that it doesn't generalize. I currently use it to solve problems that I can absolutely guarantee were not in its training set, and it generalizes well enough. I also think that one of the easiest ways to get it to generalize better would simply be through giving it synthetic data which demonstrates the process of generalizing.

I don't think what LLMs are currently doing is really generalizing, but rather:

1) Multiple occurrences of something in the dataset are mutually statistically reinforcing. This isn't generalization (abstraction) but rather reinforcement through repetition.

2) Multiple different statistical patterns are being recalled/combined in novel ways such that it seems able to "correctly" respond to things out of dataset, but really this only due to these novel combinations, not due to it having abstracted it's knowledge and applying a more general (or analogical) rule than present in it's individual training points.

> problems that I can absolutely guarantee were not in its training set

Can you share the strongest example?

Pretty much any coding problem in a unique or private codebase

When I talk to people that use copilot to improve their coding workflow, what I often hear is that copilot can replace boilerplate but not any business logic specific to the problem being solved.

It sounds like you have a different experience(copilot is useful for business logic). Do you have maybe any examples of what you mean by "any coding problems"?

Is it similar to what I've heard previously(copilot can replace boilerplate) or have LLMs actually solved business problems for you in code?

There is a difference between interpolation, which the majority of humans are performing daily with coding in private codebases, and genuine extrapolation, which is difficult to prove and difficult to find in high dimensional spaces. LLMs may not be able to easily extrapolate (and when it does it's due to high temperature), but they can interpolate extremely well, and most human growth and innovation today comes from novel interpolations, which are what LLMs are excellent at.

I asked for the strongest example the OP can share in order to evaluate their claim. If it's so obvious to the OP that generalisation is happening then it should be easy to provide a strong example, right?

Reply on news.ycombinator.com