Hacker News Reader

That's why the Chinchilla paper (given a single paragraph in the article) is so important; it gives a scaling equation that puts a limit on the effect of increasing parameters. Generally for the known transformer models the reduction in loss from having infinite parameters is significantly less that the reduction in loss from training on infinite data. Most large models are very undertrained.

My issue is less with the strategy as NN optimisaiton, and more as the OP says, a road to general intelligence.

There's an a priori "limit" on the form of any time/space complexity "intelligence" can have: it cannot be exponential.

We know that if you exhaust all infinite number of ways of describing a system by facts (/data), then you don't need intelligence. Whatever intelligence is, it's a strategy animals use for competent action in the face of their bodies not being able to exponentially expend energy to solve every trivial problem.

Exponential complexity is a killer to the project of building intelligence in every respect. The universe, a priori, cannot require an exponential space complexity description of it to be parsable: since that would require exponentially many universes.

Exponential scaling, for me (via ordinary conceptual analysis), seems a hallmark of any system not using intelligence. Exp. scaling is "the shortcut" to solving the problem without intelligence. It's the hallmark of the system not building O(log_{large number}) rich representations of the world.

Everything of interest in ML networks is occurring in the abstractions that emerge in training in deep multi-layer networks.

At the crudest level this immediately provides for more than canned lookup as asserted; analogical reasoning is a much-documented emergent property.

But analogies are merely the simplest, first-order abstraction, which are easy for humans to point at.

Inference and abstraction across multiple levels, means the behaviors of these systems is utterly unlike simple stores. One clear demonstration of this is the effective "compression" of image gen networks. They don't compress images. For lack of any better vocabulary, they understand them well enough produce them.

The hot topic is precisely whether there are boundaries to what sorts of implicit reasoning can occur through scale, and, what other architectures need to be present to effect agency and planning of the kind hacked at in traditional symbolic systems AI.

It might be worthwhile to read contemporary work to get up to speed. Things are already a lot weirder than we have had time to internalize.

Can they be said to understand the images if a style transfer model they produce is image dependent with an unstable threshold boundary? Or when they make an error similar to pareidolia all the time, seeing faces where there are none? When they do not understand how to paint even roughly fake text?

Serious answer: they can, just not as as well as we do.

The problem is not that really what they are doing, which is more akin to what e.g our own visual processing stack does...

...it's that we don't have language for a lot of the territory these systems operate in.

We are used to a world defined by very clear boundaries between agents (which e.g. have state, and agency, and carry world representations which we can infer and reason about and behave against) and tools (potentially complex but inert, at best narrowly-constrained),

these exist in the in-between where until now we only really had "animals" or maybe "institutions" but that is more of an abstract analogy.

Like say in insect, they understand but poorly—compared to us as the gold standard. And we have silos of deep specialization (language) and world knowledge beyond the purely visual. Eventually it's clear there will be a knitting together of systems which provide those, as we cobble together something that "understands" more than just the visual.

But already when you look closer it's clear they already understand some things as well as we do, or better.

Pareidolia is a fabulous word and frame, I've used the behavior of these models also to explain what hallucinations are like (and are) when on psychedelics to people who haven't experienced them.

Anyway. What does a toddler understand? Or a dog? Or...

On the bright side I am seeing language come into use before our eyes. The most established and interesting one to emerge in common use this year IMO is the use of "latent" to describe that which these things understand... they render that which is latent in their complex encodings. A great adaptation of a domain specific term of art into lay use.

Fascinating times to be sure.

The tone of this betrays a possibly more argumentative than collaborative conversation style than that which I may want to engage with further (as seems common I've noticed amongst anti-connectionists), but I did find one point intersting for discussion.

> Parameters are just the database of the system.

Would any equations parameters be considered just the database then? C in E=MC^2, 2 in a^2+b^2=c^2?

I suppose those numbers are basically a database, but the relationships (connections) they have to the other variables (inputs) represent a demonstrable truth about the universe.

To some degree every parameter in a nn is also representing some truth about the universe. How general and compact that representation is currently is not known (likely less than we'd like of both traits).

This discussion (the GP and your response) perhaps suggests that a way to evaluate the intelligence of an AI may need to be more than the generation of some content, but also citations and supporting work for that content. I guess I'm suggesting that the field could benefit from a shift towards explainability-first models.

I'm not anti-connectionist, but if I were to put myself in their shoes, I'd respond by pointing out that in E=MC^2, C is a value which directly correlates with empirical results. If all of humanity were to suddenly disappear, a future advanced civilization would re-discover the same constant, though maybe with different units. Their neural networks, on the other hand, probably would be meaningfully different.

Also, the C in E=MC^2 has units which define what it means in physical terms. How can you define a "unit" for a neural network's output?

Now, my thoughts on this are contrary to what I've said so far. Even though neural network outputs aren't easily defined currently, there's some experimental results showing neurons in neural networks demonstrating symbolic-like higher-level behavior:

https://openai.com/blog/multimodal-neurons/

Part of the confusion likely comes from how neural networks represent information -- often by superimposing multiple different representations. A very nice paper from Anthropic and Harvard delved into this recently:

https://transformer-circuits.pub/2022/toy_model/index.html

Related: Polysemanticity and Capacity in Neural Networks https://arxiv.org/abs/2210.01892

There's a very literal sense in which NN parameters are just a db. As in, it's fairly trivial to get copyrighted verbatim output from a trained NN (eg., quake source code from git co-pilot, etc.).

"Connectionists" always want to reduce everything to formulae with no natural semantics and then equivocate this with science. Science isnt mathematics. Mathematics is just a short hand for a description of the world made true by the semantics of that description.

E=mc^2 isnt true because it's a polynomial, and it doesnt mean a polynomial, and it doesnt have "polynomial properties" because it isnt about mathematics. It's about the world.

E stands for energy, m for mass, and c for a geometric constant of spacetime. If they were to stand for other properties of the world, in general, the formulae would be false.

I find this "connectionist supernaturalism" about mathematics deeply irritating, it has all the hubris and numerology of religions but wandering around in a stolen lab coat. Hence the tone.

What can one say or feel in the face of the overtaking of science by pseudoscience? It seems plausible to say now, today, more pseudoscientific papers are written than scientific ones. A generation of researchers are doing little more than analysing ink-blot patterns and calling them "models".

The insistence, without explanation, that this is a reasonable activity pushes one past tolerance on these matters. It's exasperating... from psychometrics to AI, the whole world of intellectual life has been taken over by a pseudoscientific analysis of non-experimental post-hoc datasets.

> This isnt even on the road to intelligence, let alone an instance of it.

We don't know if your database analogy is also true for the human brain.

If it isn't, then you have a point. If it is, then you probably don't.

300 GB is nothing compared to the vastness of information in the universe (hence it fitting on a disk). AI is approximating a function, and the function they are now learning to approximate is us.

From [1], with my own editing...

When comparing the difference between now and human performance

> ...[huamns] can achieve closer to 0.7 bits per character . What is in that missing >0.4?

> Well—everything! Everything that the model misses. While just babbling random words was good enough at the beginning, at the end, it needs to be able to reason our way through the most difficult textual scenarios requiring causality or commonsense reasoning... every time that it lacks the theory of mind to compress novel scenes describing the Machiavellian scheming of a dozen individuals at dinner jockeying for power as they talk...

> If we trained a model which reached that loss of <0.7, which could predict text indistinguishable from a human, whether in a dialogue ...how could we say that it doesn’t truly understand everything?

[1] https://www.gwern.net/Scaling-hypothesis