Hacker News Reader

Comment by roca | Hacker News Reader

I don't know much about ML but I wonder what the need for so much training data (apparently about 500 billion words for GPT-3) means for this approach. Humans achieve their performance levels while only ever observing a tiny fraction of that training data --- at least in word form. I see only two possibilities:

-- Human brains somehow are able to learn from all forms of sensory input and world interaction in ways that pay off in word tests. (Even then, would the information processed by an average human match the GPT-3 training corpus?)

-- The human brain has a very different architecture that lets us learn vastly more efficiently from small amounts of training data.

Is there another possibility I'm missing?

If the latter is true then that gives humans a durable advantage over GPT-3-like approaches.

View on HN

> Is there another possibility I'm missing?

Possibility: The human brain and GPT-3 are doing radically different things and aren't even comparable. GPT-3 is merely memorizing enough language to pass a turing test, whereas human brains are actually learning how to use language to communicate with other humans.

Evidence: Have GPT-3 write your work emails for a day, and then live with the consequences of that for the next week. It's going to produce text that makes sense, but only in a very particular way. You will end up with email threads where an outsider says "yeah looks like work emails. I believe this is two humans" And, that's very impressive! But your actual interlocutor who understands the full context of the conversation and relationship will legitimately worry for your mental health and maybe even contact your manager.

Conclusions:

1. Any time you invent a test, people will find ways to pass the test with flying colors but completely miss the point of the test. The turing test is no different.

2. Being able to imitate a human well enough in a five minute general english conversation is only so useful. There's a reason we don't pay people living wages to write internet comments. This isn't to say that GPT-3 is useless, though. There is certainly demand for very specialized five minute conversers that come at zero marginal cost. I'm worried.

3. We still have no clue how to even begin to approach AGI.

I think that you make an interesting point - that these transformer models can produce reasonable text only freed from context, and generally the useful thing is to produce reasonable text taking context into account.

But how far are we really from going beyond the five minute conversation you mention? What if you combine a transformer text model with a chess engine to produce a chess tutor? It doesn't seem like we are too far from a computer program that could teach chess from first principles (i.e. learn chess itself, and teach it in a meaningful way).

Maybe in combination with economic models, it could teach economics?

What else? Perhaps in combination with wikipedia it could teach subjects such as physics or biology? But it would just be regurgitating in that case, not "understanding" what is in wikipedia. We need an existing model to tie to the text generating capability. So what if you take an alpha-go-zero approach to model generation - instead of a human creating the model, it is developed through adversarial learning? Theoretically then, anything that can be formulated in a contest or game could be "learned" and modeled, and then combined with text generation to be taught. That seems pretty powerful, and not so far out of reach.

(Also, I love the idea of letting GPT reply to work emails for a day. Someone should set up some experiments to actually do that, and find out what happens. I bet that even though it would be a disaster, we would learn a ton.)

Yeah, I think that sort of stuff is exactly the future of computing. Just because GPT-{N+1} isn't an AGI doesn't mean that GPT-{N+1} won't cause an exciting step-change in what's possible with computing.

> Someone should set up some experiments to actually do that, and find out what happens. I bet that even though it would be a disaster, we would learn a ton.

Ha! Agreed! Unfortunately I do too much soft external-facing stuff these days to do this myself :(. Someone who interacts mostly internally with engineers/techy types who might appreciate the experiment should totally do this.

You may find this interesting https://www.reddit.com/r/SubSimulatorGPT2

> Have GPT-3 write your work emails for a day

This almost certainly won't work with GPT-3, but it might with GPT-5, GPT-10, or GPT-20.

It depends how data hungry those models get. It might be the case that to train GPT-10 you actually need more data than there is available in the universe, so you have no chance of training it.

Learning from sensory input and world interaction PLUS language is a huge thing. Like teaching a dog to sit. The dog learns about the world with all sorts of sensory input, it interacts with the world, and it hears you speak a ton. But it'll never learn what "sit" means without an explicit combination of language and real world interaction. We do the same. Feedback loops are important too. Teaching your dog to sit, the words and actions you take depend on what the dog's doing. Teaching a baby to talk, your words and actions depend on what the kid's doing and saying, and maybe even asking and answering questions.

Current language models learn language from how we use it, but only in relation to more language. Hopefully, we can eventually get some good data on how we use language in relation to the real world too. I can't imagine where you'd get a massive dataset like that though, shy of having robots out in the world acting like babies.

> Human brains somehow are able to learn from all forms of sensory input and world interaction in ways that pay off in word tests. (Even then, would the information processed by an average human match the GPT-3 training corpus?)

Yes, absolutely humans learn from cross modalities. I haven't seen much work on attempting this in neural networks, but cross modal prediction is known to work well.

> Even then, would the information processed by an average human match the GPT-3 training corpus?

I'd be surprised if it didn't. The visual cortex processes around 8.75 megabits per second[1]. Assuming eyes are open around 16 hours a day that is 63 GB/day of information just from the eyes.

Assuming 500 billion words in the GPT training set, and 5 characters per word on average and 1 byte characters that is 100 GB training set or a bit under 2 days of information for a human.

Now the two aren't directly comparable, but humans gather a lot of information.

[1] https://www.newscientist.com/article/dn9633-calculating-the-...

If this explanation is true, it suggests a very important experiment would be to develop AI models that train on video and (much less than 500 billion words of) text and then tackle these test problems that GPT-3 is evaluated on.

Indeed.

It's not entirely obvious how to do this though. There needs to be a common training representation and that's pretty hard.

In videos, the sequence of frames is important, but each frame is an image. In text the sequence of characters is important.

Maybe something that can accept sequences (of some length..) of bytes where the bytes maybe an image or maybe a single character might work. But unified representations of images and words for training is probably the first step towards that.

based on the current state of the art and the research problems needed to solve it's probably 3 years before we are in a position to contemplate the kind of training suggested here.

Another very important aspect is that humans interact with the world they are observing. It isn’t just passive processing of data. Training a model on video may help over text alone, but the interactivity is still missing.

Probably a combination of factors not a single one. In addition to the ones you mentioned: GPT is also much smaller than a brain (fewer parameters), so it can't form higher-level concepts that might help it to organize things better. And the human brain has priors about the world baked in through evolution something that GPT sort of avoids by working on arbitrary token sequences.

I considered the "pretrained via genetics" possibility but I don't see how that could make much difference. It doesn't look like the genome has enough information to encode really large amounts of brain structure details.

It's not really pretraining, more of an architecture optimized for the problem-space the brain is dealing with. There are specialized regions for different tasks while a transformer is uniform. Being uniform makes it easier to scale, but it probably means while being generic it's also kind of inefficient so it needs way more training and parameters than an optimized architecture.

Brains are also quite energy-efficient, which makes parallel search of architectures a lot cheaper. A small rodent having a handful of offspring every year is equivalent to building new supercomputers with specialized hardware beyond current capabilities on a tiny budget and then using that to train a model and seeing whether it's better. Researchers don't have that luxury so they have to compromise by running a very generic model that is flexible but inefficient.

That makes sense but it kind of falls into my "brain has a better architecture" possible explanation. What's interesting is that the thrust of the GPT approaches is "architecture doesn't matter while we can just keep building bigger models".

It would be surprising to me if AGI is achievable via two such different pathways. I don't know this area so I'm willing to be surprised.

> That makes sense but it kind of falls into my "brain has a better architecture" possible explanation. What's interesting is that the thrust of the GPT approaches is "architecture doesn't matter while we can just keep building bigger models".

Those are not mutually exclusive. Architecture doesn't matter asymptotically but it does matter for task-specific performance for any concrete complexity budget. Compare to Big-O behavior of algorithms (asymptotes) vs. doing hardware specific optimizations (cache-friendliness, hand-rolled assembly, outsourcing specific parts to ASICs...).

OA is doing the former, looking at asymptotes. Nature has done that too (mammalian brains scale from rodents to humans) but also has applies all kinds of task-specific optimizations. Compare the cerebral cortex and cerebellum for example, the latter acts similar to an application accelerator but if it doesn't work software fallbacks are possible albeit slower.

> It would be surprising to me if AGI is achievable via two such different pathways.

Have you considered cephalopod or bird brains?

There's a lot of stuff, language for example, that's mostly genetic. Humans aren't learning these things from scratch, generally they're just filling in some blanks and helping guide the development of the brain.

see: https://en.wikipedia.org/wiki/Universal_grammar

I generally agree, but I think there is something else going on too. Conceptualization is highly flexible and comes in really early. Little kids ask "why?" a lot. Why do they? There must be some energy minimization reason for it. I don't think we're going to get AGI without it being able to ask some questions and get some answers that fundamentally update its understanding of the world.

There's also human's ability to probe the real world and understand and create new constraints. Putting a square block into a circle shaped hole teaches you new things.

> -- The human brain has a very different architecture that lets us learn vastly more efficiently from small amounts of training data.

I think this is right, but its misleading perhaps to identify this as 'efficiency' in a computational sense - we're talking trillions of extra parameters derived in the brain from the tiny fraction of training data

It is not entirely clear how it would be correct to calculate the amount of data required by a human language center to learn that much. Do you count all the random thoughts in someone's mind? What if those thoughts, and also some subconscious processess are constantly running in that network?

Input to the human (like audible words said in front of them), and input to the network (whatever data that part of the brain receives from anywhere, both from outside but also from other parts of the brain) are two very distinct things.

> gives humans a durable advantage over GPT-3-like approaches.

I think it is pretty clear to everyone involved that there are, empirically, so far, numerous very important and obvious and deep advantages that humans have over a GPT-3. This doesn't mean that GPT-2/3 aren't a tremendous achievement in AI.

> Humans achieve their performance levels while only ever observing a tiny fraction of that training data

Individual humans only ever observe a tiny fraction of that training data, but our brains are not the products of solely our individual experiences. We come out of the womb with a massive amount of evolutionarily trained, genetically encoded weights in our neural network.

Where are those weights encoded? It doesn't seem like the genome has enough bits to encode a lot of weights.

I agree with other people in this thread that structure/architecture is encoded, but not weights.

> Humans achieve their performance levels while only ever observing a tiny fraction of that training data

This is really the key detail and the hole in Gwern's argument that AGI is around the corner. You can't just compare the result. You also have to look at what it took to train the model and at what the model is actually doing.

If you look at GPT-3's output, it only superficially makes sense. Is there evidence of true understanding or is it just a really really good text generator?

Regardless of AGI though, I do think that models like this will eventually mean the end of social media and perhaps all wide open discourse on the Internet. When this stuff gets easy and cheap enough that spammers and propagandists can use it, it's over. How much money in compute/storage would it take to train GPT-3 to advocate for Donald Trump or Joe Biden all day long, or to shill products, or to just generate superficially comprehensible text to execute a kind of denial of service attack on a community?

I don't buy the "we have GPT-3, therefore we may soon have artificial general intelligence" (AGI) notion.

AGI might happen tomorrow; it might happen in decades, in centuries, or never. GPT-3 is basically a straightforward scaling of GPT-2, but I see no evidence that simply scaling GPT-2 or GPT-3 will lead to AGI. The problem is we don't know what else is needed.

Maybe the end of anonymous ungated social media, yes. We can still talk to humans and sources we know and trust. Probably that's what we need to do already.

given the idiosyncrasies of GPT style text generation, the cost of detecting generated text is likely to be orders of magnitude lower than the cost of generating the text. Such detection may also capture low-quality content on the internet and prove a boon for high-quality commentary.

Whether humans have a genetic language facility is not yet known.

It is also kind of a moot point. Whether or not there is a genetic component, it is relevant only as far it is cashed out in physical brain matter and its arrangement.

Reply on news.ycombinator.com